PromQL: derive the default subquery step from the data instead of a hardcoded 15 seconds

Start with NodeEvaluationRangeGetter, PrometheusQueryEvaluationSettings, and PrometheusHTTPProtocolAPI to trace how the current defaults are selected. Benchmark the proposed data scan against wide and narrow selectors, then use the results to decide the supported scope and whether the default remains opt-in.

Describe the situation

NodeEvaluationRangeGetter carries two hardcoded defaults:

/// By default the lookback period is 5 minutes.
constexpr const Int64 DEFAULT_INSTANT_SELECTOR_WINDOW_SECONDS = 5 * 60;

/// The default subquery step is 15 seconds.
constexpr const Int64 DEFAULT_SUBQUERY_STEP_SECONDS = 15;

Both can be overridden through PrometheusQueryEvaluationSettings, but only the first one ever is: PrometheusHTTPProtocolAPI sets instant_selector_window from Prometheus's lookback_delta. Nothing sets default_subquery_step, so a subquery written without a step — max_over_time(rate(m[5m])[30m:]) — always evaluates on a 15-second grid, whatever the data looks like.

For Prometheus that constant is not arbitrary: the default subquery step is the configured global evaluation interval, which the server knows because it did the scraping. ClickHouse has no such configuration to read, so the 15 seconds is a guess. On data scraped every 60 seconds it produces four grid points per sample, and on data scraped every second it produces one point per fifteen samples.

The information needed to do better is in the data. A per-series scrape interval can be estimated from the gaps between consecutive samples. Taking a quantile of those deltas rather than the mean or the minimum is robust to both missed scrapes and bursts; around the 0.6 quantile works well in practice.

Proposal

Three shapes, and I would like an opinion on which (if any) is wanted before writing code:

1. Use the detected interval as the default subquery step, behind a setting, off by default. No change to query syntax and no divergence from PromQL semantics — it replaces a ClickHouse-invented constant with a value derived from the data it is evaluating.
2. The same for the instant-selector lookback. Riskier: that one already mirrors Prometheus's lookback_delta, so changing it silently changes which samples an instant query sees.
3. Accept a range function with no window, rate(m), substituting the detected interval. This is the shape that most improves the first-use experience, and also the most invasive: it is a change to the PromQL grammar, since rate(m) is a parse error in Prometheus. The PromQL layer here has tracked Prometheus so far, including its experimental functions (limitk, ts_of_min_over_time), with the divergence documented, so accepting syntax Prometheus rejects looks like a project decision rather than an implementation detail.

The cost, which I think decides it

The estimate is not free. Per series it means something on the order of

quantile(0.6)(timestamp - lagInFrame(timestamp) OVER (PARTITION BY id ORDER BY timestamp))

which is a pass over the timestamps of every selected series before the outer grid can be built — a second phase in the generated SQL. On a wide selector that can easily cost more than the wrong grid step ever did.

So whichever shape is chosen, it needs a measurement rather than an assertion: how much the extra pass costs against how much a data-derived step saves, on a selector wide enough to hurt. If the answer is that it only pays off on narrow selectors, a setting that is off by default is the honest outcome, and option 3 becomes hard to justify at all since it would silently add that pass to the simplest query a user can write.

Questions

• Is a data-derived default wanted here, or is the hardcoded step considered fine and the answer "write the step you want"?
• If it is wanted, is option 1 the right scope to start with?
• Is there appetite for the grammar extension in option 3, or should PromQL syntax compatibility stay strict?

Happy to implement whichever is preferred, with the measurement above.