apache / apache/beam

writeProtos() should allow a user to specify message attributes

Open
#18,469 0 comments 0 reactions 0 assignees View on GitHub
core improvement java P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 5h
Merged PRs (30d)
204

Description

when using PubsubIO.writeProtos(protoMessage), the PubsubMessage is created in the background and there is no way to specify the attributes of the message. This makes the method useless if a user wants to then use the withTimestampAttribute() or withIdAttribute() of the writer with anything other than the default timestamp or ID (a common use case).

As a workaround the user can manually create the PubsubMessage, but this basically requires duplication of the writeProtos() logic, which is brittle in the case that the encoding logic changes.

Imported from Jira [BEAM-2451](https://issues.apache.org/jira/browse/BEAM-2451). Original Jira may contain additional context.
Reported by: berkoben.

Contributor guide

Open the contributing guide

Research direction

Start by locating PubsubIO.writeProtos(protoMessage) and the writer methods withTimestampAttribute() and withIdAttribute(). Trace how PubsubMessage is constructed and how related writer options are tested. Done means users can specify message attributes without duplicating the protobuf encoding logic, with coverage for non-default timestamp or ID attributes.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
40/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.