elastic / elastic/observability-migration-platform

Grafana "All" template variable surfaces as ".*" in the migrated ES|QL control (cosmetic "Incompatible selections" label)

Open
#286 0 comments 0 reactions 0 assignees View on GitHub
asset:dashboards enhancement program:grafana-engine quality source:grafana workstream:translator
Dominant language
Python
Stars
6
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
23

Description

## Summary

When a Grafana **query** template variable with `includeAll: true` (or an `All`/multi `current`) is migrated, its Kibana ES|QL values control is given the regex match-all sentinel `.*` as its `selected_options` default. This is **functionally correct** — it binds the panel's `WHERE RLIKE ?var` to `.*` and renders every series on first load — but it has a cosmetic UX wrinkle:

- The control **displays `.*`** as its selected value instead of a friendlier "All".
- The control's dropdown shows an EUI **"Incompatible selections (0)"** group header (count `0`, nothing listed) because `.*` is not one of the values returned by the control's `VALUES_FROM_QUERY` list.

This is a **display-only** issue. No data is wrong, and first-load rendering is correct.

## Evidence (live serverless 9.5.0)

Migrated a minimal dashboard: `$service` query var (`multi: true, includeAll: true, current: All`) + panel `sum(rate(http_requests_total{service=~"$service"}[$__rate_interval])) by (service, status_code)`.

Uploaded saved-object control config (ground truth):

```json
{
"control_type": "VALUES_FROM_QUERY",
"title": "Service",
"variable_name": "service",
"variable_type": "values",
"esql_query": "FROM metrics-prometheus-default | WHERE http_requests_total IS NOT NULL AND service IS NOT NULL | STATS count = COUNT(*) BY service | SORT service ASC | KEEP service | LIMIT 1000",
"selected_options": [".*"],
"single_select": true
}
```

Clean first load (**brand-new dashboard id, no prior interaction**): control value `.*`, legend shows all 9 series (backend/checkout/frontend × 200/404/500). Panel `/_query` bound `params: [{"service":".*"}]`. Selecting `backend` re-dispatched `params: [{"service":"backend"}]` and collapsed to the 3 backend series — interactive filtering works.

> Note during investigation: a reload initially showed `backend` pre-selected. That was **stale per-user unsaved dashboard state keyed by the dashboard id** (survived `localStorage`/`sessionStorage` clears and delete/recreate of the same id) — not the saved default. A fresh id confirmed the real default is `.*`.

## Where it comes from

- `observability_migration/adapters/source/grafana/panels.py`: `_MATCH_ALL_SELECTION = ".*"`, `_variable_default_selection()` (returns `.*` for `All`/unknown), `_build_esql_param_control()`.
- `_matcher_to_esql()` / `_collect_regex_default_param_names()` deliberately emit `RLIKE ?var` so the `.*` default matches everything — this part is correct and should be preserved.

## Proposed direction (needs its own scoped investigation)

Explore a Kibana-native representation of "All / no filter" for an ES|QL values control that avoids showing `.*` and the empty "Incompatible selections" group, **without regressing** the verified `RLIKE ?var` match-all binding. Candidate options to evaluate against a live target:

1. A friendlier selected label / placeholder for the match-all sentinel.
2. Defaulting to "all available options selected" (requires confirming how a multi-select ES|QL control binds `?var` and whether `RLIKE` vs `IN (?var)` stays faithful to Grafana's regex-alternation expansion).
3. Any control flag that permits an out-of-list custom value without the incompatible-group header.

Whatever the approach, it must be validated live (generated YAML → uploaded saved object → clean-state browser session) and must refresh the affected control/snapshot goldens intentionally.

## Acceptance

- Migrated `All`/`includeAll` template-variable control no longer displays the raw `.*` sentinel and drops the empty "Incompatible selections (0)" header on a clean first load.
- Default view still renders **all** series on first load; selecting a concrete value still filters (verified live, not just offline).
- Snapshot/control goldens refreshed intentionally.

Contributor guide

Open the contributing guide

Research direction

Start in observability_migration/adapters/source/grafana/panels.py, reading _MATCH_ALL_SELECTION, _variable_default_selection(), _build_esql_param_control(), and the _matcher_to_esql() path. Investigate Kibana-native representations for an All selection while preserving the RLIKE ?var match-all behavior, then validate generated YAML, the uploaded saved object, and a clean browser session. Done means no raw .* or empty incompatible group, correct first-load and filtering behavior, and intentionally refreshed control/snapshot goldens.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability, tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.