apache / apache/pulsar

[Go Functions] forwardSourceMessageProperty is not implemented, so source message properties are dropped

Open
#26,408 0 comments 0 reactions 0 assignees View on GitHub
area/function type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

Master Issue: #26404

### Search before reporting

- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Motivation

`SinkSpec.forwardSourceMessageProperty` tells the runtime to copy the input message's properties onto the output message. The Go runtime does not implement it — `git grep -in "ForwardSourceMessageProperty" -- 'pulsar-function-go/**/*.go'` returns nothing outside the generated `pb` package.

The Python runtime implements it in `process_result` and has tests pinning both directions (`TestPropertiesForwarding.test_forwards_properties` and `test_do_not_forward_properties` in `test_python_instance.py`), and the Java runtime implements it too.

The effect is that `--forward-source-message-property` is accepted, reported back by `functions get`, and produces output messages carrying none of the source properties. For a chain of functions this loses correlation ids, trace context and any other metadata a previous stage attached — and it does so silently, so the loss surfaces downstream as missing data rather than as a configuration error.

### Solution

Copy the source message's properties onto the output message when the flag is set, in the Go equivalent of Python's `process_result`. Pulsar's own `__pfn_input_topic__` and `__pfn_input_msgid__` properties are set separately and should keep their current behaviour.

Worth confirming in review: whether the properties are copied before or after the runtime's own reserved keys, so a source message carrying a `__pfn_*` key cannot overwrite them. Python appends the runtime keys after the forwarded ones.

### Alternatives

None worth taking. The Go client's `ProducerMessage` has a `Properties map[string]string` field, so this is a small change.

### Anything else?

Verified against `origin/master`.

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start in the Go runtime under pulsar-function-go/**/*.go and compare its result-processing path with Python's process_result. Review test_python_instance.py, especially TestPropertiesForwarding.test_forwards_properties and test_do_not_forward_properties, then add equivalent Go coverage if the relevant test entry point is found. Done means source properties are forwarded only when the flag is set, while runtime __pfn_* keys retain their current behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.