Allow plugins to register and report metrics
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
Current plugins have no way to register metrics(Corresponding MetricsRegistery utils code is in pinot-common), hence no metrics are really published along with each component.
We need to implement a way to dynamically extract metrics from plugins, so plugins can report metrics based on their own needs.
E.g. filesystem plugins should be able to report the API counter/latency metrics
Kafka plugins should be able to report topic partition information and consumer lag etc.
Contributor guide
Research direction
Start by examining the MetricsRegistery utility code in pinot-common and how current plugins are loaded and exposed by Pinot components. Define how plugins register metrics and how those metrics are dynamically extracted and published, using filesystem API counters/latencies and Kafka topic-partition and consumer-lag metrics as examples. Done means plugin-provided metrics are registered and reported by each component.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100