Increase the amount of active work for bounded pipelines using the Dataflow Runner using the portability framework
Open
dataflow
improvement
P3
portability
runners
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
Process one work item per CPU core for bounded pipelines on the Dataflow runner when using the portability framework.
Imported from Jira [BEAM-3402](https://issues.apache.org/jira/browse/BEAM-3402). Original Jira may contain additional context.
Reported by: lcwik.
Contributor guide
Research direction
Start by locating the Dataflow Runner implementation and portability framework entry points for bounded pipelines. Inspect how active work is currently allocated, then verify that processing uses one work item per CPU core and add or update coverage for this behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100