elastic / elastic/observability-migration-platform
Grafana: legacy `singlestat` panel unit (top-level `format`) is ignored — percent and byte tiles render raw numbers
- Dominant language
- Python
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 23
Description
## Summary
On pre-`fieldConfig` Grafana dashboards (`schemaVersion` ≤ ~26), a `singlestat` panel stores
its display unit at the panel's **top level** (`panel["format"]`) rather than in
`fieldConfig.defaults.unit`. The single-value path reads only the modern location, so the unit
is silently dropped and migrated tiles render raw numbers.
The unit *mapping* already exists and is correct (`GRAFANA_UNIT_TO_YAML` in
`observability_migration/targets/kibana/emit/display.py` maps `percentunit` → percent and
`decbytes` → bytes). Only the **read location** is missing, so this is a small, contained fix.
Scope note: legacy `graph` panels in the same dashboard are **not** affected — their
`yaxes[0].format` is read correctly (`Cluster Mem Capacity` emits `"format": {"type": "bits"}`).
The gap is specific to the `singlestat` / single-value path.
## Reproduction
Dashboard: [grafana.com 13332 "kube-state-metrics-v2"](https://grafana.com/grafana/dashboards/13332-kube-state-metrics-v2/)
(`schemaVersion: 26`, 24 `singlestat` + 12 `graph` + 6 `table-old` panels).
```bash
obs-migrate migrate --source grafana --input-mode files \
--input-dir ./input --output-dir ./out --assets dashboards \
--data-view metrics-k8s.prometheus-parity --esql-index metrics-k8s.prometheus-parity \
--field-profile auto --es-url "$ES_URL" --es-api-key "$KEY"
obs-migrate upload --artifact-dir out/dashboards --kibana-url "$KB_URL" --kibana-api-key "$KEY"
```
The dashboard migrates cleanly (38/42 panels, no warning on any affected tile), so nothing
signals the loss.
## Evidence — same data, both stacks fed by one Prometheus
| Panel | Source `format` | Grafana | Migrated Kibana |
|---|---|---|---|
| Cluster Pod Requested | `percentunit` | **4.55%** | **0.045** |
| Cluster CPU Requested | `percentunit` | **41.5%** | **0.415** |
| Cluster Memory Requested | `percentunit` | **24.1%** | **0.241** |
| Memory Requested By Containers | `decbytes` | **10.4 GB** | **10,368,319,488** |
The underlying ES|QL is correct — running the migrated query directly returns
`value=0.04545`, `0.415`, `0.2414`, `1.037e10`, matching Grafana exactly. This is purely a
display-format regression, not a query problem.
The emitted panel carries no format at all:
```json
{
"type": "metric",
"title": "Cluster Pod Requested",
"data_source": { "type": "esql", "query": "PROMQL index=… | STATS value = LAST(value, step)" },
"metrics": [ { "type": "primary", "column": "value", "label": "Cluster Pod Requested" } ],
"hide_title": true
}
```
…while a `graph` panel in the same dashboard does carry one, confirming the asymmetry:
```json
{ "format": { "type": "bits" } }
```
## Root cause
`observability_migration/adapters/source/grafana/panels.py`:
```python
def _panel_field_defaults(panel):
defaults = ((panel or {}).get("fieldConfig") or {}).get("defaults") or {}
return defaults if isinstance(defaults, dict) else {}
```
For these panels `fieldConfig.defaults` is literally `{"custom": {}}` while the unit sits at
`panel["format"]`. Every consumer of `_panel_field_defaults(...).get("unit")` therefore sees
an empty unit — this also disables the dependent `percentunit` 0–1 → 0–100 rescaling and the
threshold/color-domain conversion (`_percentunit_values_scaled_to_percent_points`,
`_metric_display_domain`), so the `thresholds: "80,90"` on these tiles cannot map correctly
either.
In dashboard 13332 **all 24** `singlestat` panels have no `fieldConfig.defaults.unit`;
4 carry a meaningful legacy `format` (3 × `percentunit`, 1 × `decbytes`).
## Expected
`_panel_field_defaults` (or its single-value callers) should fall back to the legacy location
when `fieldConfig.defaults.unit` is absent — `panel["format"]` and `panel["decimals"]` for
`singlestat` (and `table-old`) — so that `percentunit` renders `4.55%` and `decbytes` renders
`10.4 GB`, matching the source and the treatment `graph` already gets.
## Why it matters
`singlestat` dashboards are still extremely common on grafana.com (13332 alone is referenced
6417 times). A percentage tile that reads `0.045` instead of `4.55%` looks like a broken
migration to an operator even though the number is right, and nothing in the migration report
flags it.
Contributor guide
Research direction
Start in observability_migration/adapters/source/grafana/panels.py by reading _panel_field_defaults and its single-value callers, then inspect the display mapping in observability_migration/targets/kibana/emit/display.py. Run the provided dashboard migration or a focused equivalent with legacy singlestat data; done means percentunit and decbytes formats, including dependent scaling and thresholds, are preserved for affected panels without changing graph behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- grafana, python
- Domain
- observability-sre, tooling
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100