Need to add an unique id to MetricsSet or ExecutionPlanMetricsSet
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and *why* for this feature, in addition to the *what*)
The is useful for Ballista Scheduler to post merge the metrics of the plan operators from all the Executor task results.
Today, this is a missing part in Ballista. The task results returned from the executors do not report back any of the plan node execution metrics to Scheduler. After adding the report back feature, the metrics need to be correctly merged, need a way to uniquely identify which metrics belong to which plan operator and only the metrics belong to the same plan node can be merged
in the Scheduler side.
@alamb
Please share your thoughts and advice.
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
Contributor guide
Research direction
Start by tracing MetricsSet and ExecutionPlanMetricsSet through Ballista executor task results and the Scheduler. Determine how metrics for plan operators are identified and merged; done means metrics from the same plan node can be uniquely identified and merged correctly in the Scheduler.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100