Question: do all server types of map and reduce in pynumaflow support user metadata?

[tmenjo]

I'd like my vertices to propagate user-generated small metadata items associated with payload from source through map and reduce to sink. The items are, for example, index, height, or width of each frame of a video. I also want some items to be appended by map or reduce vertices.

However, regarding map and reduce, it seems that certain server types (batchmapper, mapstreamer, and so on) in pynumaflow does not support user metadata. I looked into _dtypes.py of each server type at latest main (a83067c) but could not find user_metadata in some cases. Does such server, especially mapstreamer and accumulator, can handle user metadata?

Map

(1) Does Datum have user_metadata?
(2) Does Message have user_metadata?

Server type	(1)	(2)
batchmapper	No	No
mapper	Yes	Yes
mapstreamer	No	No

Reduce

(1) and (2) are the same as above.

Server type	(1)	(2)
accumulator	No	No
reducer	No	No
reducestreamer	No	No

[vaibhavtiwari33]

Your analysis is correct, currently metadata propagation is limited to source, transformer, mapper and sink components across all SDKs.

We should include metadata propagation to the pending components cc: @yhl25 @vigith