Delegate beam metrics to runners
Open
flink
P3
runners
spark
sub-task
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
Delegate beam metrics to runners to avoid forwarding updates, i.e., extract updates from beam metrics and commit updates in runners.
For Flink/Spark runners, we can reference metrics within runner's metrics system in beam pipelines and update them directly without forwarding.
Imported from Jira [BEAM-2604](https://issues.apache.org/jira/browse/BEAM-2604). Original Jira may contain additional context.
Reported by: Cody.
Subtask of issue #18470
Contributor guide
Research direction
The issue names no files or tests; start by tracing beam metrics into the Flink and Spark runners and reading the runner metrics systems. Done means runner metrics can be referenced and updated directly without forwarding metric updates.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- observability
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100