elastic / elastic/observability-migration-platform

Re-check whether variable filters can stay on native PROMQL

Open
#319 1 comment 0 reactions 0 assignees View on GitHub
asset:dashboards program:grafana-engine source:grafana
Dominant language
Python
Stars
6
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
23

Description

## Summary

Panels whose PromQL uses a Grafana template variable in a label matcher (e.g. `{device=\"$device_filtered\"}`) are routed to **native ES|QL** instead of **native PROMQL**, even with `--translation-mode native`.

That fallthrough is intentional today (issue #230 / PR #236): params inside an opaque `PROMQL …` command were observed to stay unbound in Kibana (`Parameter [?var] value not found`), so migration surfaces the filter as `WHERE … ?var` instead.

But the platform stack that #64 assumed is now in place:

| Layer | Ticket | Claimed outcome |
| --- | --- | --- |
| Elasticsearch | [elastic/elasticsearch#148620](https://github.com/elastic/elasticsearch/issues/148620) / [PR#149744](https://github.com/elastic/elasticsearch/pull/149744) | `?params` in PromQL label matchers |
| Kibana | [elastic/kibana#271215](https://github.com/elastic/kibana/issues/271215) / [PR#271244](https://github.com/elastic/kibana/pull/271244) | Variable controls bind into PromQL label matchers |
| Migration | #64 / PR #133 | `$var` → `?var` (and `=` → `=~` for multi/All), keep native PROMQL |

**Ask:** re-verify on current Kibana/ES (e.g. 9.5) whether a Lens **dashboard** panel with native PROMQL + a matching control actually binds. If yes, #230's guard is stale and we should keep these panels on native PROMQL (per #64). If no, document the remaining Kibana gap and keep the fallthrough.

Related: #316 already notes the PROMQL skip as background for the ES|QL `TBUCKET` bug; this issue is specifically about **whether that skip is still correct**.

## Expected vs actual (Issue 1)

Grafana source:

```promql
node_disk_read_bytes_total{device="$device_filtered"}
```

(`device_filtered` is multi-select.)

**Expected (native PROMQL, #64 + dashboard timing from #318):**

```esql
PROMQL index=metrics-* value=(node_disk_read_bytes_total{device=~?device_filtered})
```

**Actual today:** native PROMQL skipped → ES|QL TS path (e.g. `TBUCKET(…)` + `WHERE` / params for the device control).

Code guard: `panels.py` returns `None` from the native PROMQL path when `_promql_label_matcher_has_template_variable(expr)` (comment cites #230).

## Reproduction

1. Migrate a Grafana panel with `{label=\"$var\"}` (or `=~` / multi) under native PromQL translation.
2. Confirm the uploaded panel is ES|QL, not `PROMQL … value=(…{label=~?var})`.
3. Manually paste the expected PROMQL above into a Lens dashboard panel with a `device_filtered` control on Kibana 9.5+ and record whether the control binds (screenshot).

```bash
obs-migrate migrate \
--source grafana \
--assets dashboards \
--input-mode files \
--input-dir /path/to/folder/with/dashboard.json \
--kibana-url \"$KIBANA_URL\" \
--kibana-api-key \"$KIBANA_API_KEY\" \
--es-url \"$ES_URL\" \
--es-api-key \"$ES_API_KEY\" \
--upload
```

Minimal source (Issue 1 panel + vars)

```json
{
"title": "Issue 1",
"uid": "afst21a0rak8wf",
"panel": {
"title": "Panel Title",
"type": "timeseries",
"targets": [
{
"datasource": {
"type": "prometheus",
"uid": "prometheus"
},
"disableTextWrap": false,
"editorMode": "builder",
"expr": "node_disk_read_bytes_total{device=\"$device_filtered\"}",
"fullMetaSearch": false,
"includeNullMetadata": true,
"legendFormat": "__auto",
"range": true,
"refId": "A",
"useBackend": false
}
],
"interval": null
},
"templating": [
{
"name": "device_all",
"type": "query",
"multi": true,
"includeAll": null,
"query": {
"qryType": 1,
"query": "label_values(device)",
"refId": "PrometheusVariableQueryEditor-VariableQuery"
}
},
{
"name": "device_filtered",
"type": "query",
"multi": true,
"includeAll": null,
"query": {
"qryType": 1,
"query": "label_values(node_disk_read_bytes_total{device!=\"nbd1\"},device)",
"refId": "PrometheusVariableQueryEditor-VariableQuery"
}
}
]
}
```

Contributor guide

Open the contributing guide

Research direction

Start in panels.py at the native PROMQL guard using _promql_label_matcher_has_template_variable, then run the provided obs-migrate command with the minimal Grafana dashboard. Manually test the expected PROMQL in a Lens dashboard on Kibana 9.5+ with the device_filtered control. Done means recording whether the control binds and either removing the stale fallthrough or documenting the remaining Kibana gap.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, python
Domain
observability, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.