elastic / elastic/observability-migration-platform

PROMQL panels should follow dashboard time resolution, not hardcode 50 buckets

Open
#318 3 comments 0 reactions 0 assignees View on GitHub
asset:dashboards bug phase:3 program:grafana-engine quality source:grafana workstream:translator
Dominant language
Python
Stars
6
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
23

Description

## Summary

After migrating Grafana timeseries panels that use **auto** resolution (`$__rate_interval` / dashboard time range), the native PROMQL path hardcodes `buckets=50`:

```esql
PROMQL index=metrics-* start=?_tstart end=?_tend buckets=50 value=(...)
```

Grafana did not hardcode a bucket count — resolution follows the dashboard time range.

**In a Kibana dashboard panel**, the time range is inferred from the time picker. Bare PROMQL works (confirmed on local Kibana/ES **9.5.0-SNAPSHOT**), e.g. on `Request total`:

```esql
PROMQL index=metrics-* value=(node_disk_read_bytes_total)
```

Outside a dashboard (raw `_query`), timing args are still required (`step` or `start`+`end`[+`buckets`]). Migration of **dashboard panels** should not bake in `start`/`end`/`buckets=50`.

Also: `step` and `buckets` are mutually exclusive. When Grafana sets panel `interval` (e.g. `Request total (1h step)`), emit `step=1h` — not `buckets=50`.

## Reproduction

```bash
obs-migrate migrate \
--source grafana \
--assets dashboards \
--input-mode files \
--input-dir /path/to/folder/with/dashboard.json \
--kibana-url "$KIBANA_URL" \
--kibana-api-key "$KIBANA_API_KEY" \
--es-url "$ES_URL" \
--es-api-key "$ES_API_KEY" \
--upload
```

Source Grafana dashboard JSON

```json
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": {
"type": "grafana",
"uid": "-- Grafana --"
},
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"id": 12,
"links": [],
"panels": [
{
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green"
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 8,
"x": 0,
"y": 0
},
"id": 3,
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "11.6.0",
"targets": [
{
"editorMode": "code",
"expr": "node_disk_read_bytes_total",
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "Request total ",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green"
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 8,
"x": 8,
"y": 0
},
"id": 1,
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "11.6.0",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"editorMode": "code",
"expr": "rate(node_disk_read_bytes_total[$__rate_interval])",
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "Request rate (auto)",
"type": "timeseries"
},
{
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisBorderShow": false,
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"barWidthFactor": 0.6,
"drawStyle": "line",
"fillOpacity": 0,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"insertNulls": false,
"lineInterpolation": "linear",
"lineWidth": 1,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "auto",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green"
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 8,
"x": 16,
"y": 0
},
"id": 2,
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"hideZeros": false,
"mode": "single",
"sort": "none"
}
},
"pluginVersion": "11.6.0",
"targets": [
{
"editorMode": "code",
"expr": "increase(node_disk_read_bytes_total[$__rate_interval])",
"legendFormat": "__auto",
"range": true,
"refId": "A"
}
],
"title": "Request increase (auto)",
"type": "timeseries"
}
],
"preload": false,
"schemaVersion": 41,
"tags": [],
"templating": {
"list": []
},
"time": {
"from": "now-6h",
"to": "now"
},
"timepicker": {},
"timezone": "browser",
"title": "Issue 2",
"uid": "efst5tc0vnpj4e",
"version": 6
}
```

### What we see after migration

Every panel query includes hardcoded `buckets=50`, e.g. on `Request rate (auto)`:

```esql
PROMQL index=metrics-* start=?_tstart end=?_tend buckets=50 value=(rate(node_disk_read_bytes_total))
```

Source Grafana queries used auto lookback via `$__rate_interval` (or no range window on the raw counter), with no fixed bucket count.

## Acceptance criteria

1. **Do not emit `start`/`end`/`buckets=50`** on native PROMQL dashboard panels. Kibana infers the time range from the dashboard time picker. Bare `PROMQL index=… value=(…)` is enough for auto panels.

2. **When Grafana sets panel `interval`**, emit `step=` with that value (still no `buckets`). Example: `Request total (1h step)`, `Request rate (auto) (1h step)`, and `Request increase (auto) (1h step)` should get `step=1h`.

3. **Rate lookback**: `$__rate_interval` → windowless `rate`/`increase` (e.g. `Request rate (auto)`); explicit windows stay explicit (e.g. `Request rate (auto) (30m rate)` keeps `[30m]`).

4. **Check in Kibana**: auto panels have no timing args; the `(1h step)` panels are coarser than their auto siblings; no `buckets=50` in the query editor.

## Out of scope / related gap

Full Grafana `$__rate_interval` scrape flooring (`max(step + scrape_interval, 4 × scrape_interval)`) needs a Prometheus `scrape_interval` we do not currently emit — separate from dropping `buckets=50`.

Contributor guide

Open the contributing guide

Research direction

Start at the Grafana-to-native-PROMQL migration path invoked by the obs-migrate command and compare its generated ES|QL with the dashboard JSON in this issue. Verify auto panels omit timing arguments, interval panels emit step, and rate or increase windows follow the stated rules. Validate the resulting queries in Kibana, ensuring no buckets=50 remains.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.