elastic / elastic/observability-migration-platform

Grafana: topk()/bottomk() migrates as snapshot, not time-series (temporal loss)

Open
#59 0 comments 0 reactions 0 assignees View on GitHub
asset:dashboards phase:8 program:grafana-engine source:grafana stability workstream:translator
Dominant language
Python
Stars
6
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
23

Description

#### Problem statement

Grafana's `topk(N, expr)` and `bottomk(N, expr)` functions are used to display the top or bottom N time-series lines over a selected time window — for example, "the 5 busiest pods by CPU over the last hour" shown as 5 continuous trend lines. The migration tool currently translates these expressions to a valid ES|QL query, but the result is semantically different: instead of N time-series lines, the Kibana panel shows N individual data points representing the most recent value per series, sorted by that value. The temporal history — the defining characteristic of these panels — is lost.

---

#### Known cases

The ES|QL translation path is used for all `topk()`/`bottomk()` expressions (native PromQL blocks them as unsupported constructs).

| Expression | Native PromQL path | ES\|QL path (current) | ES\|QL path (after fix) |
|---|---|---|---|
| `topk(5, rate(process_cpu_seconds_total[5m]))` | Blocked — `topk` unsupported | ⚠️ migrated with warnings — single snapshot, no time-series | ⚠️ migrated with warnings — N series by most recent value; time-series history not replicated |
| `bottomk(5, rate(process_cpu_seconds_total[5m]))` | Blocked — `bottomk` unsupported | ⚠️ migrated with warnings — single snapshot, no time-series | ⚠️ migrated with warnings — N series by most recent value; time-series history not replicated |

Example dashboards:
- [Calico: Felix](https://grafana.com/grafana/dashboards/12175) — *Felix restarts/hr (top N)*, *CPU Usage (top N)*, *Open file descriptors (top N)*
- [VictoriaMetrics - vmalert](https://grafana.com/grafana/dashboards/14950) — *Top groups avg evaluation duration*, *Top active alerts*

Note: `topk($variable_limit, expr)` (template variable used as the limit argument) is a separate parse-failure bug tracked in [#58](https://github.com/elastic/observability-migration-platform/issues/58).

---

#### Verification

**Check 1 — Routing**

```
topk(5, rate(process_cpu_seconds_total[5m])):
can_use_native_promql = False
_native_promql_has_distinct_metric_arithmetic = False
→ routing: esql_translation
```

**Check 2 — Current CLI behaviour**

```
feasibility = feasible
warnings = ['topk() without group labels: collapsed to single-series top N;
add preferred_group_labels hint for per-series breakdown']
esql_query:
TS metrics-*
| WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend
| WHERE process_cpu_seconds_total IS NOT NULL
| STATS _bucket_value = AVG(RATE(process_cpu_seconds_total, 5m))
BY time_bucket = TBUCKET(5 minute)
| SORT time_bucket ASC
| STATS value = LAST(_bucket_value, time_bucket)
| SORT value DESC
| LIMIT 5
```

The generated query takes the last bucket value per series and returns the top 5 by most-recent value. A single aggregate point is returned per series rather than the full time-series history.

**Check 3 — ES|QL function check**

```
ROW val = 2.0 | EVAL r = TOPK(val, 5)
→ HTTP 400: Unknown function [TOPK], did you mean [top]?
```

ES|QL does not have a `TOPK` scalar function. The current CLI correctly avoids it and uses `SORT … LIMIT` instead. Producing the correct time-series top-N result requires a two-pass query: first identify the top N label sets across the full time range, then fetch their complete time buckets.

**Check 4 — Grafana baseline**

```
topk(5, rate(process_cpu_seconds_total[5m])):
STATUS: success | 1 series | value: [1779905664, '0.0086']
Labels: {instance: "localhost:9090", job: "prometheus"}

bottomk(5, rate(process_cpu_seconds_total[5m])):
STATUS: success | 1 series | value: [1779905664, '0.0086']
```

Both expressions return valid data. With only 1 series available in the local Prometheus, topk and bottomk both return that same series — ranking becomes meaningful with multiple series in production.

---

#### Expected behaviour

ES|QL does not support a true time-series top-N query (no equivalent to PromQL's per-step ranking). After the fix, the panel remains `migrated_with_warnings` but with an honest warning that clearly describes what the output actually is: N series ranked by their most recent value, not continuous trend lines. Users can then decide whether this approximation is acceptable. The current warning (`'topk() without group labels: collapsed to single-series top N'`) is replaced with an explicit message such as: `'topk()/bottomk() shows N series ordered by most recent value only — time-series history from Grafana is not replicated'`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.