elastic / elastic/observability-migration-platform

Native PROMQL: support on()/ignoring()/group_* when the target supports vector matching

Open
#440 0 comments 0 reactions 1 assignee Claimed by @giorgi-imerlishvili-elastic View on GitHub
asset:dashboards capabilities enhancement program:grafana-engine source:grafana workstream:translator
Dominant language
Python
Stars
6
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
23

Description

## Context

Elasticsearch now supports PromQL vector matching (`on(...)`, `ignoring(...)`, `group_left`, `group_right`) on **Serverless** and **Stack 9.6** ([elasticsearch#155634](https://github.com/elastic/elasticsearch/pull/155634)).

`obs-migrate` should emit those panels as native `PROMQL index=…` on capable targets, and keep today's ES|QL fallback elsewhere. (`or` with `on`/`ignoring` is still open in [elasticsearch#158181](https://github.com/elastic/elasticsearch/issues/158181) — out of scope here.)

## Example dashboard

| # | Panel | Expected on capable ES |
|---|---|---|
| 1 | Control (no matcher) | `PROMQL index=…` (already works) |
| 2 | `on()` aligned ratio | `PROMQL index=…` |
| 3 | `ignoring()` | `PROMQL index=…` |
| 4 | `on()` + `group_left` | `PROMQL index=…` |

**Today:** panel 1 is native PROMQL; panels 2–4 are not (ES|QL or placeholders depending on target fields).

Dashboard uid `on-ignoring-eval`. Replace the Prometheus datasource uid before import.

Full Grafana dashboard JSON

```json
{
"uid": "on-ignoring-eval",
"title": "PromQL on/ignoring eval",
"tags": ["obs-migrate-eval", "on-ignoring"],
"timezone": "browser",
"schemaVersion": 39,
"version": 1,
"refresh": "",
"time": { "from": "now-1h", "to": "now" },
"panels": [
{
"id": 1,
"type": "timeseries",
"title": "Control (no matcher)",
"gridPos": { "h": 8, "w": 24, "x": 0, "y": 0 },
"datasource": { "type": "prometheus", "uid": "REPLACE_ME" },
"targets": [
{
"refId": "A",
"expr": "rate(node_cpu_seconds_total[5m])",
"legendFormat": "__auto",
"datasource": { "type": "prometheus", "uid": "REPLACE_ME" }
}
],
"fieldConfig": { "defaults": {}, "overrides": [] },
"options": {}
},
{
"id": 2,
"type": "timeseries",
"title": "on() aligned ratio",
"gridPos": { "h": 8, "w": 24, "x": 0, "y": 8 },
"datasource": { "type": "prometheus", "uid": "REPLACE_ME" },
"targets": [
{
"refId": "A",
"expr": "sum by (instance) (rate(node_cpu_seconds_total{mode=\"user\"}[5m])) / on(instance) sum by (instance) (rate(node_cpu_seconds_total[5m]))",
"legendFormat": "__auto",
"datasource": { "type": "prometheus", "uid": "REPLACE_ME" }
}
],
"fieldConfig": { "defaults": {}, "overrides": [] },
"options": {}
},
{
"id": 3,
"type": "timeseries",
"title": "ignoring()",
"gridPos": { "h": 8, "w": 24, "x": 0, "y": 16 },
"datasource": { "type": "prometheus", "uid": "REPLACE_ME" },
"targets": [
{
"refId": "A",
"expr": "sum(rate(node_network_receive_bytes_total[5m])) / ignoring(device) sum(rate(node_network_transmit_bytes_total[5m]))",
"legendFormat": "__auto",
"datasource": { "type": "prometheus", "uid": "REPLACE_ME" }
}
],
"fieldConfig": { "defaults": {}, "overrides": [] },
"options": {}
},
{
"id": 4,
"type": "timeseries",
"title": "on() + group_left",
"gridPos": { "h": 8, "w": 24, "x": 0, "y": 24 },
"datasource": { "type": "prometheus", "uid": "REPLACE_ME" },
"targets": [
{
"refId": "A",
"expr": "node_hwmon_temp_celsius * on(chip) group_left(chip_name) node_hwmon_chip_names",
"legendFormat": "__auto",
"datasource": { "type": "prometheus", "uid": "REPLACE_ME" }
}
],
"fieldConfig": { "defaults": {}, "overrides": [] },
"options": {}
}
]
}
```

```bash
obs-migrate migrate \
--source grafana --input-mode files \
--input-dir ./on_ignoring_eval \
--output-dir ./on_ignoring_out \
--assets dashboards \
--es-url "$ELASTICSEARCH_ENDPOINT" --es-api-key "$KEY" \
--kibana-url "$KIBANA_ENDPOINT" --kibana-api-key "$KEY" \
--upload
```

## Acceptance criteria

- [ ] At migrate time, detect whether this Elasticsearch accepts vector matching
- [ ] If Elasticsearch accepts it: emit native `PROMQL index=…` for `on` / `ignoring` / `group_left` / `group_right` (example panels 2–4)
- [ ] If Elasticsearch does not accept it: keep ES|QL fallback
- [ ] Do not enable `or on(…)` / `or ignoring(…)` — [elasticsearch#158181](https://github.com/elastic/elasticsearch/issues/158181) is still open
- [ ] Ship in an `obs-migrate` version aligned with **ES 9.6** (capability already on **Serverless** + **9.6**); older Stack without the capability stays on ES|QL

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.