SDKs should measure time spent for active element
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 204
Description
Measuring the time spent in active element for each ptransform will be important in progress reporting as outlined in https://s.apache.org/beam-fn-api-progress-reporting. Each SDK needs to provide this the Runner to be able to achieve efficient dynamic work rebalancing.
Imported from Jira [BEAM-2820](https://issues.apache.org/jira/browse/BEAM-2820). Original Jira may contain additional context.
Reported by: vikasrk.
Contributor guide
Research direction
Start with the Beam Fn API progress-reporting specification linked in the issue and identify the SDK entry points responsible for ptransform execution and progress reporting. Define how each SDK can provide active-element time to the Runner, then verify that the result supports efficient dynamic work rebalancing.
Written by the indexing model from the issue text.
Assessment
- Domain
- data-engineering, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100