elastic / elastic/observability-migration-platform
[Grafana] Multi-target panel fusion keeps only one target; support multi-series overlays incl. attribute-split and cross-data-stream sources
- Dominant language
- Python
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 23
Description
## Problem statement
Panels that overlay multiple PromQL targets (a primary series plus reference/comparison series, or several related metrics) currently keep one target and drop the rest. The panel lands `migrated_with_warnings` with a generic "Panel has N PromQL targets but only 1 could be migrated" reason, so the loss is disclosed, but the comparison intent (primary vs reference) is gone.
On OTel targets this is not only a fusion problem, it is a composition problem, because the dropped targets can be:
- **attribute-split**: the reference series is the same target metric distinguished by an attribute value, or
- **cross-data-stream**: the primary and reference series live in different target data streams, which a single ES|QL panel query cannot trivially join.
## Proposed change (general capability)
- For panels with multiple compatible targets, keep the overlay: emit each feasible target as its own series rather than only the first.
- Where a single ES|QL layer cannot host all targets, prefer keeping the primary plus its reference series, using per-series queries / Lens layers when the sources are attribute-split or in different data streams.
- Make the degrade reason specific (name the dropped series and why: incompatible target, attribute-split not supported yet, cross-data-stream composition), not just a count.
- For summary/quota tables, capture the multi-column intent as a redesign task with per-column ES|QL sketches.
Interacts with #309 (attribute-split and per-metric data-stream mapping) and #310 (some dropped targets are recording rules with no target field).
## Source / target impact
Grafana source → Kibana/OTel target. Affects any multi-target overlay or summary-table panel, common in infra/k8s dashboards.
## Additional context
Multi-target fusion / target selection in `observability_migration/adapters/source/grafana/panels.py` and `translate.py`. Attribute-split and cross-data-stream constraints verified against Elastic's managed `kubernetes_otel` dashboards; see comment for a concrete example. Related: #309, #310, #271.
Contributor guide
Research direction
Start in observability_migration/adapters/source/grafana/panels.py and translate.py, then read the concrete dashboard example in the issue and related issues #309 and #310. Trace multi-target fusion and target selection first. Done means compatible overlays retain feasible series, attribute-split and cross-data-stream cases have an explicit handling or degradation reason, and summary-table redesign work records per-column ES|QL sketches.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, grafana, prometheus, python
- Domain
- data, observability-sre, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100