Add support for overriding PTransforms with multiple outputs
Open
core
improvement
P3
python
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
Currently PTransform override feature (Python) doesn't work for PTransforms with multiple outputs.
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/pipeline.py#L225
This is needed to support SDFs with side outputs.
Imported from Jira [BEAM-3477](https://issues.apache.org/jira/browse/BEAM-3477). Original Jira may contain additional context.
Reported by: chamikara.
Contributor guide
Research direction
Start at sdks/python/apache_beam/pipeline.py around line 225 and trace how PTransform overrides are handled for outputs. Confirm the current limitation with multiple-output transforms, then verify that overrides also work for SDFs with side outputs; the issue provides no test path or additional files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100