apache / apache/beam

"Retroactively logging runtime parameters" guide simply doesn't work.

Open
#20,524 0 comments 0 reactions 0 assignees View on GitHub
bug documentation P3 website
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

The Java version of the "[Pipeline option patterns](https://beam.apache.org/documentation/patterns/pipeline-options/)" article simply does not work.

When creating a PipelineOptions inside of a DoFn, all get methods that return ValueProvider will return null instead.  The closest working example I could find was from Google's guide on [ValueProviders in Dataflow templates|#using-valueprovider-in-your-functions].  The pattern involves passing the ValueProvider into a DoFn class's constructor, storing it, then later accessing it at runtime.

 

Imported from Jira [BEAM-10700](https://issues.apache.org/jira/browse/BEAM-10700). Original Jira may contain additional context.
Reported by: mtruscello.

Contributor guide

Open the contributing guide

Research direction

Start with the Java example in the linked Pipeline option patterns article and compare it with the referenced ValueProviders guide. Update the example so runtime parameters are available in the DoFn, then verify that the documented pattern works when PipelineOptions is not created inside the DoFn.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.