Outputting to an undeclared side output TupleTag should throw
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
Currently some runners (perhaps all of them?..), when you output to a side output tag that wasn't passed to ParDo.withOutputTags, simply drop the data on the floor (or increase some aggregators).
Silent data dropping is never okay, and it's never okay to output to an undeclared tag. So instead this should throw.
Imported from Jira [BEAM-1979](https://issues.apache.org/jira/browse/BEAM-1979). Original Jira may contain additional context.
Reported by: jkff.
Contributor guide
Research direction
Start by tracing ParDo.withOutputTags and the runner behavior when a TupleTag is not declared. Compare how the affected runners handle such output and look for existing tests around side outputs. Done means undeclared side-output data is rejected with an exception rather than silently dropped, with coverage for the relevant runner behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100