Create a transform to assign an index when reading data from built in I/O transforms
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
As a beam user
when using any of the existing beams I/O transforms
I would like to add an index to each line read as part of the transform.
As spark has zipWithIndex to assign an index when reading files with the beam being abstraction layer for may runners. I would expect this feature should be added to beam
Imported from Jira [BEAM-7692](https://issues.apache.org/jira/browse/BEAM-7692). Original Jira may contain additional context.
Reported by: saidivi.
Contributor guide
Research direction
Start by reviewing the existing built-in I/O transforms and the original Jira issue BEAM-7692 for the missing context. Define how indexing should work across the requested I/O transforms, then verify that each line read receives an index and that the behavior fits Beam's batch and streaming model.
Written by the indexing model from the issue text.
Assessment
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100