Expose segment reload status as JMX Metric
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
It would be very helpful for operation purpose to have the ability to get quick overview on the reload status of a table.
All I could find for now is the `GET /segments/{tableName}_OFFLINE/metadata?type=OFFLINE`. As I'm exploring its output, I realize that the reload status is specific to each individual segments. While providing a startree status there would be a good step forward, a global status would greatly improve the operation experience.
### Output from metadata endpoint
```
"myTable_OFFLINE_1602111600000_1602111600000_2670" : {
"segmentName" : "myTable_OFFLINE_1602111600000_1602111600000_2670",
...
"columns" : [ ],
"indexes" : {
"column1" : {
"bloom-filter" : "NO",
"dictionary" : "YES",
"forward-index" : "YES",
"inverted-index" : "NO",
"null-value-vector-reader" : "NO",
"range-index" : "NO"
},
"column2" : {
"bloom-filter" : "NO",
"dictionary" : "YES",
"forward-index" : "YES",
"inverted-index" : "NO",
"null-value-vector-reader" : "NO",
"range-index" : "NO"
},
}
}
```
## Exposing reload status as JMX Metrics
In addition to a global status, exposing the individual reload status as well as the global status would be very helpful in regards to configuration alerts on a prometheus stack.
Contributor guide
Research direction
The issue identifies GET /segments/{tableName}_OFFLINE/metadata?type=OFFLINE as the current entry point for per-segment reload information; start by tracing that status and Pinot's JMX metric exposure. Done means JMX provides both a global table reload status and individual segment reload statuses suitable for operational alerts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100