elastic / elastic/observability-migration-platform
Grafana "All" template variable surfaces as ".*" in the migrated ES|QL control (cosmetic "Incompatible selections" label)
- Dominant language
- Python
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 23
Description
## Summary
When a Grafana **query** template variable with `includeAll: true` (or an `All`/multi `current`) is migrated, its Kibana ES|QL values control is given the regex match-all sentinel `.*` as its `selected_options` default. This is **functionally correct** — it binds the panel's `WHERE RLIKE ?var` to `.*` and renders every series on first load — but it has a cosmetic UX wrinkle:
- The control **displays `.*`** as its selected value instead of a friendlier "All".
- The control's dropdown shows an EUI **"Incompatible selections (0)"** group header (count `0`, nothing listed) because `.*` is not one of the values returned by the control's `VALUES_FROM_QUERY` list.
This is a **display-only** issue. No data is wrong, and first-load rendering is correct.
## Evidence (live serverless 9.5.0)
Migrated a minimal dashboard: `$service` query var (`multi: true, includeAll: true, current: All`) + panel `sum(rate(http_requests_total{service=~"$service"}[$__rate_interval])) by (service, status_code)`.
Uploaded saved-object control config (ground truth):
```json
{
"control_type": "VALUES_FROM_QUERY",
"title": "Service",
"variable_name": "service",
"variable_type": "values",
"esql_query": "FROM metrics-prometheus-default | WHERE http_requests_total IS NOT NULL AND service IS NOT NULL | STATS count = COUNT(*) BY service | SORT service ASC | KEEP service | LIMIT 1000",
"selected_options": [".*"],
"single_select": true
}
```
Clean first load (**brand-new dashboard id, no prior interaction**): control value `.*`, legend shows all 9 series (backend/checkout/frontend × 200/404/500). Panel `/_query` bound `params: [{"service":".*"}]`. Selecting `backend` re-dispatched `params: [{"service":"backend"}]` and collapsed to the 3 backend series — interactive filtering works.
> Note during investigation: a reload initially showed `backend` pre-selected. That was **stale per-user unsaved dashboard state keyed by the dashboard id** (survived `localStorage`/`sessionStorage` clears and delete/recreate of the same id) — not the saved default. A fresh id confirmed the real default is `.*`.
## Where it comes from
- `observability_migration/adapters/source/grafana/panels.py`: `_MATCH_ALL_SELECTION = ".*"`, `_variable_default_selection()` (returns `.*` for `All`/unknown), `_build_esql_param_control()`.
- `_matcher_to_esql()` / `_collect_regex_default_param_names()` deliberately emit `RLIKE ?var` so the `.*` default matches everything — this part is correct and should be preserved.
## Proposed direction (needs its own scoped investigation)
Explore a Kibana-native representation of "All / no filter" for an ES|QL values control that avoids showing `.*` and the empty "Incompatible selections" group, **without regressing** the verified `RLIKE ?var` match-all binding. Candidate options to evaluate against a live target:
1. A friendlier selected label / placeholder for the match-all sentinel.
2. Defaulting to "all available options selected" (requires confirming how a multi-select ES|QL control binds `?var` and whether `RLIKE` vs `IN (?var)` stays faithful to Grafana's regex-alternation expansion).
3. Any control flag that permits an out-of-list custom value without the incompatible-group header.
Whatever the approach, it must be validated live (generated YAML → uploaded saved object → clean-state browser session) and must refresh the affected control/snapshot goldens intentionally.
## Acceptance
- Migrated `All`/`includeAll` template-variable control no longer displays the raw `.*` sentinel and drops the empty "Incompatible selections (0)" header on a clean first load.
- Default view still renders **all** series on first load; selecting a concrete value still filters (verified live, not just offline).
- Snapshot/control goldens refreshed intentionally.
Contributor guide
Research direction
Start in observability_migration/adapters/source/grafana/panels.py, reading _MATCH_ALL_SELECTION, _variable_default_selection(), _build_esql_param_control(), and the _matcher_to_esql() path. Investigate Kibana-native representations for an All selection while preserving the RLIKE ?var match-all behavior, then validate generated YAML, the uploaded saved object, and a clean browser session. Done means no raw .* or empty incompatible group, correct first-load and filtering behavior, and intentionally refreshed control/snapshot goldens.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability, tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100