Source.getDefaultOutputCoder() - add a version that allows access to a CoderRegistry
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
The Source API's getDefaultOutputCoder currently takes no parameters. This means that you can't access the coder registry to ask for coders based on the type.
The PTransform API's version of getDefaultOutputCoder has 3 different form, which do allow access to the coder registry if necessary.
However, while working on BEAM-1040 (Hadoop IO connector), we realized that a source does not always know it's output type, and thus will need to calculate the coder at run time. To do this, it would be helpful to have a reference to the CoderRegistry - we should look at what we're doing on PTransform and decide if we should port that over onto Source.
Imported from Jira [BEAM-1171](https://issues.apache.org/jira/browse/BEAM-1171). Original Jira may contain additional context.
Reported by: sisk.
Contributor guide
Research direction
Start by comparing the Source API's getDefaultOutputCoder with the three PTransform variants, focusing on how each exposes CoderRegistry. Review the BEAM-1040 Hadoop IO connector context and the original BEAM-1171 discussion. Done means deciding the appropriate Source API shape and documenting or implementing registry access for sources whose output type is determined at runtime.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100