ClickHouse / ClickHouse/ClickHouse
Support Prometheus-compatible monotonicity repair in quantilePrometheusHistogram
- Dominant language
- C++
- Stars
- 49.9k
- Forks
- 9k
- Avg merge
- 21h 32m
- Merged PRs (30d)
- 515
Description
### Problem
`quantilePrometheusHistogram` performs PromQL-style interpolation for classic histogram buckets, but it does not implement the monotonicity repair performed by Prometheus `histogram_quantile()`.
Classic histogram buckets are cumulative and should be monotonically increasing. In practice, values produced by `rate()` can temporarily become non-monotonic because bucket time series may have missing, delayed, or non-atomic scrapes, independently handled counter resets, partial ingestion, or small floating-point discrepancies after aggregation.
Prometheus handles this by ignoring tiny relative differences and then raising any decreasing bucket value to the previous value. The ClickHouse function instead requires monotonic input and silently produces a plausible but incorrect result when that precondition is violated.
The plural `quantilesPrometheusHistogram` function uses the same implementation and has the same behavior.
### Reproduction
Verified with ClickHouse 25.12.5:
```sql
SELECT quantilePrometheusHistogram(0.5)(
bucket_upper_bound,
cumulative_bucket_value
)
FROM VALUES(
'bucket_upper_bound Float64, cumulative_bucket_value Float64',
(1, 2),
(2, 1),
(inf, 3)
);
```
Current result:
```text
2
```
Prometheus repairs the cumulative values from `[2, 1, 3]` to `[2, 2, 3]`. With three observations, the p50 rank is 1.5, and interpolation in the first bucket `[0, 1]` produces:
```text
0.75
```
### Expected behavior
Ideally, `quantilePrometheusHistogram` and `quantilesPrometheusHistogram` should apply the same repair as Prometheus:
1. Ignore tiny relative differences between consecutive buckets using the Prometheus tolerance.
2. Apply a running maximum so cumulative bucket values are non-decreasing.
3. Perform quantile interpolation on the repaired values.
If automatic repair is not considered appropriate for a ClickHouse aggregate, detecting non-monotonic input and returning `NaN` or an error would be safer than returning a valid-looking incorrect quantile.
### Motivation
Users moving PromQL-compatible histogram quantile workloads to this native aggregate can otherwise receive silently different percentile results for the same bucket data.
### References
- ClickHouse documentation: https://clickhouse.com/docs/sql-reference/aggregate-functions/reference/quantilePrometheusHistogram
- Original ClickHouse implementation: https://github.com/ClickHouse/ClickHouse/pull/86294
- Prometheus behavior: https://prometheus.io/docs/prometheus/latest/querying/functions/#histogram_quantile
Contributor guide
Research direction
Start with the quantilePrometheusHistogram implementation linked through the original ClickHouse pull request, then compare its behavior with Prometheus histogram_quantile and run the supplied VALUES query. Done means both quantilePrometheusHistogram and quantilesPrometheusHistogram handle non-monotonic cumulative buckets compatibly, including the documented 0.75 result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, prometheus, sql
- Domain
- analytics, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100