ClickHouse / ClickHouse/ClickHouse

Support Prometheus-compatible monotonicity repair in quantilePrometheusHistogram

Open
#110,624 0 comments 0 reactions 0 assignees View on GitHub
comp-aggregate-functions external
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

### Problem

`quantilePrometheusHistogram` performs PromQL-style interpolation for classic histogram buckets, but it does not implement the monotonicity repair performed by Prometheus `histogram_quantile()`.

Classic histogram buckets are cumulative and should be monotonically increasing. In practice, values produced by `rate()` can temporarily become non-monotonic because bucket time series may have missing, delayed, or non-atomic scrapes, independently handled counter resets, partial ingestion, or small floating-point discrepancies after aggregation.

Prometheus handles this by ignoring tiny relative differences and then raising any decreasing bucket value to the previous value. The ClickHouse function instead requires monotonic input and silently produces a plausible but incorrect result when that precondition is violated.

The plural `quantilesPrometheusHistogram` function uses the same implementation and has the same behavior.

### Reproduction

Verified with ClickHouse 25.12.5:

```sql
SELECT quantilePrometheusHistogram(0.5)(
bucket_upper_bound,
cumulative_bucket_value
)
FROM VALUES(
'bucket_upper_bound Float64, cumulative_bucket_value Float64',
(1, 2),
(2, 1),
(inf, 3)
);
```

Current result:

```text
2
```

Prometheus repairs the cumulative values from `[2, 1, 3]` to `[2, 2, 3]`. With three observations, the p50 rank is 1.5, and interpolation in the first bucket `[0, 1]` produces:

```text
0.75
```

### Expected behavior

Ideally, `quantilePrometheusHistogram` and `quantilesPrometheusHistogram` should apply the same repair as Prometheus:

1. Ignore tiny relative differences between consecutive buckets using the Prometheus tolerance.
2. Apply a running maximum so cumulative bucket values are non-decreasing.
3. Perform quantile interpolation on the repaired values.

If automatic repair is not considered appropriate for a ClickHouse aggregate, detecting non-monotonic input and returning `NaN` or an error would be safer than returning a valid-looking incorrect quantile.

### Motivation

Users moving PromQL-compatible histogram quantile workloads to this native aggregate can otherwise receive silently different percentile results for the same bucket data.

### References

- ClickHouse documentation: https://clickhouse.com/docs/sql-reference/aggregate-functions/reference/quantilePrometheusHistogram
- Original ClickHouse implementation: https://github.com/ClickHouse/ClickHouse/pull/86294
- Prometheus behavior: https://prometheus.io/docs/prometheus/latest/querying/functions/#histogram_quantile

Contributor guide

Open the contributing guide

Research direction

Start with the quantilePrometheusHistogram implementation linked through the original ClickHouse pull request, then compare its behavior with Prometheus histogram_quantile and run the supplied VALUES query. Done means both quantilePrometheusHistogram and quantilesPrometheusHistogram handle non-monotonic cumulative buckets compatibly, including the documented 0.75 result.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, prometheus, sql
Domain
analytics, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.