elastic / elastic/observability-migration-platform

Numeric control values break the template-variable guardrail, so the panel fails to compile

Open
#353 0 comments 0 reactions 1 assignee Claimed by @giorgi-imerlishvili-elastic View on GitHub
asset:dashboards bug program:grafana-engine quality source:grafana stability workstream:translator
Dominant language
Python
Stars
6
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
23

Description

## Summary

The template-variable guardrail the translator emits for multi-select controls fails to compile whenever the bound control values are numeric. The panel renders a red error box instead of a chart.

Grafana community dashboard [9852 "node-exporter disk graphs"](https://grafana.com/grafana/dashboards/9852-stians-disk-graphs/) has a `CPU` template variable populated from `label_values(node_cpu_seconds_total, cpu)`, so its values are `0` and `1`. The migrated `IO Wait per core` panel fails with:

```
Unexpected error from Elasticsearch: verification_exception - Found 2 problems
line 2:33: second argument of [MV_CONTAINS(?CPU, ".*")] must be [integer], found value [".*"] type [keyword]
line 2:60: second argument of [MV_CONTAINS(?CPU, labels.cpu)] must be [integer], found value [labels.cpu] type [keyword]
```

The emitted guard clause is:

```esql
| WHERE (MV_COUNT(?CPU) == 0 OR MV_CONTAINS(?CPU, ".*") OR MV_CONTAINS(?CPU, labels.cpu))
```

When the selected option values all look numeric, Kibana binds `?CPU` as an **integer** multi-value parameter. Both `MV_CONTAINS` arms then fail type-checking — one against the string literal `".*"`, one against the keyword field `labels.cpu` — and because this is a verification (compile-time) error, the whole query fails. The `MV_COUNT(?CPU) == 0` short-circuit does not help; the statement never compiles.

This is not specific to CPU indices. Any Grafana variable whose label values are numeric hits it: CPU/core IDs, ports, PIDs, HTTP status codes, queue or shard numbers, disk numbers.

## Environment

- Branch `feat/curated-dashboard-packs` (PR #346) at `5db742f`
- Kibana / Elasticsearch `9.5.0-SNAPSHOT`
- Source: Grafana 11.x serving community dashboard 9852, Prometheus + `node_exporter`, 15s scrape
- Target: `metrics-node.prometheus-default`, `--field-profile prometheus_native`
- No curated pack applies to 9852, so this is the general translator path

## Reproduction

```bash
obs-migrate migrate \
--source grafana \
--input-mode api \
--grafana-url "$GRAFANA_URL" \
--field-profile prometheus_native \
--es-url "$ES_URL" \
--es-api-key "$ES_API_KEY" \
--kibana-url "$KIBANA_URL" \
--kibana-api-key "$KIBANA_API_KEY" \
--validate --upload --ensure-data-views
```

Then, in the uploaded dashboard:

1. Open it. The `CPU` control defaults to all options selected, including the synthetic `.*` entry. The panel renders, because the presence of `.*` keeps the parameter a keyword list.
2. Deselect `.*`, leaving only `0, 1` — a normal operator action, since `.*` is the tool's stand-in for Grafana's "All".
3. Reload the dashboard.
4. `IO Wait per core` now shows the `verification_exception` above instead of a chart.

## Why the existing gates miss it

`--validate` reported `13 queries: 13 passed, 0 failed` for this dashboard, and the pre-upload verification gate was `8 Green / 5 Yellow / 0 Red`. Validation binds the parameter differently from a live control, so the type conflict never appears offline. Only a browser session with a real control selection reproduces it — the interaction-audit case described in `AGENTS.md`.

## Suggested fix

Make the guardrail type-stable regardless of how Kibana infers the control parameter type. Options, roughly in order of preference:

- Coerce both sides explicitly, e.g. compare `TO_STRING` of the parameter elements against the keyword field, so a numeric binding cannot change the comparison type.
- Force the control's option list to keyword by casting in the control query (it already does `MV_APPEND(".*", labels.cpu)`, which is keyword-typed at generation time but does not survive Kibana's re-inference of the selected values).
- Failing both, detect numeric-valued label variables at translation time and emit a different filter shape rather than one that cannot compile.

Whatever the fix, the `.*` sentinel and the field comparison need to agree on type with the bound parameter, not just with each other.

## Related

- #345 — native PROMQL control params not forwarded into label matchers (different path, same "controls do not bind cleanly" theme)
- Found while manually testing PR #346 with dashboard 9852

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.