elastic / elastic/observability-migration-platform

Grafana: legacy `singlestat` panel unit (top-level `format`) is ignored — percent and byte tiles render raw numbers

Open Beginner friendly
#374 1 comment 0 reactions 0 assignees View on GitHub
asset:dashboards bug program:grafana-engine source:grafana
Dominant language
Python
Stars
6
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
23

Description

## Summary

On pre-`fieldConfig` Grafana dashboards (`schemaVersion` ≤ ~26), a `singlestat` panel stores
its display unit at the panel's **top level** (`panel["format"]`) rather than in
`fieldConfig.defaults.unit`. The single-value path reads only the modern location, so the unit
is silently dropped and migrated tiles render raw numbers.

The unit *mapping* already exists and is correct (`GRAFANA_UNIT_TO_YAML` in
`observability_migration/targets/kibana/emit/display.py` maps `percentunit` → percent and
`decbytes` → bytes). Only the **read location** is missing, so this is a small, contained fix.

Scope note: legacy `graph` panels in the same dashboard are **not** affected — their
`yaxes[0].format` is read correctly (`Cluster Mem Capacity` emits `"format": {"type": "bits"}`).
The gap is specific to the `singlestat` / single-value path.

## Reproduction

Dashboard: [grafana.com 13332 "kube-state-metrics-v2"](https://grafana.com/grafana/dashboards/13332-kube-state-metrics-v2/)
(`schemaVersion: 26`, 24 `singlestat` + 12 `graph` + 6 `table-old` panels).

```bash
obs-migrate migrate --source grafana --input-mode files \
--input-dir ./input --output-dir ./out --assets dashboards \
--data-view metrics-k8s.prometheus-parity --esql-index metrics-k8s.prometheus-parity \
--field-profile auto --es-url "$ES_URL" --es-api-key "$KEY"
obs-migrate upload --artifact-dir out/dashboards --kibana-url "$KB_URL" --kibana-api-key "$KEY"
```

The dashboard migrates cleanly (38/42 panels, no warning on any affected tile), so nothing
signals the loss.

## Evidence — same data, both stacks fed by one Prometheus

| Panel | Source `format` | Grafana | Migrated Kibana |
|---|---|---|---|
| Cluster Pod Requested | `percentunit` | **4.55%** | **0.045** |
| Cluster CPU Requested | `percentunit` | **41.5%** | **0.415** |
| Cluster Memory Requested | `percentunit` | **24.1%** | **0.241** |
| Memory Requested By Containers | `decbytes` | **10.4 GB** | **10,368,319,488** |

The underlying ES|QL is correct — running the migrated query directly returns
`value=0.04545`, `0.415`, `0.2414`, `1.037e10`, matching Grafana exactly. This is purely a
display-format regression, not a query problem.

The emitted panel carries no format at all:

```json
{
"type": "metric",
"title": "Cluster Pod Requested",
"data_source": { "type": "esql", "query": "PROMQL index=… | STATS value = LAST(value, step)" },
"metrics": [ { "type": "primary", "column": "value", "label": "Cluster Pod Requested" } ],
"hide_title": true
}
```

…while a `graph` panel in the same dashboard does carry one, confirming the asymmetry:

```json
{ "format": { "type": "bits" } }
```

## Root cause

`observability_migration/adapters/source/grafana/panels.py`:

```python
def _panel_field_defaults(panel):
defaults = ((panel or {}).get("fieldConfig") or {}).get("defaults") or {}
return defaults if isinstance(defaults, dict) else {}
```

For these panels `fieldConfig.defaults` is literally `{"custom": {}}` while the unit sits at
`panel["format"]`. Every consumer of `_panel_field_defaults(...).get("unit")` therefore sees
an empty unit — this also disables the dependent `percentunit` 0–1 → 0–100 rescaling and the
threshold/color-domain conversion (`_percentunit_values_scaled_to_percent_points`,
`_metric_display_domain`), so the `thresholds: "80,90"` on these tiles cannot map correctly
either.

In dashboard 13332 **all 24** `singlestat` panels have no `fieldConfig.defaults.unit`;
4 carry a meaningful legacy `format` (3 × `percentunit`, 1 × `decbytes`).

## Expected

`_panel_field_defaults` (or its single-value callers) should fall back to the legacy location
when `fieldConfig.defaults.unit` is absent — `panel["format"]` and `panel["decimals"]` for
`singlestat` (and `table-old`) — so that `percentunit` renders `4.55%` and `decbytes` renders
`10.4 GB`, matching the source and the treatment `graph` already gets.

## Why it matters

`singlestat` dashboards are still extremely common on grafana.com (13332 alone is referenced
6417 times). A percentage tile that reads `0.045` instead of `4.55%` looks like a broken
migration to an operator even though the number is right, and nothing in the migration report
flags it.

Contributor guide

Open the contributing guide

Research direction

Start in observability_migration/adapters/source/grafana/panels.py by reading _panel_field_defaults and its single-value callers, then inspect the display mapping in observability_migration/targets/kibana/emit/display.py. Run the provided dashboard migration or a focused equivalent with legacy singlestat data; done means percentunit and decbytes formats, including dependent scaling and thresholds, are preserved for affected panels without changing graph behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
grafana, python
Domain
observability-sre, tooling
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.