elastic / elastic/observability-migration-platform
[Grafana] Add a source→OTel metric-name mapping layer (otel profile currently passes metric names through verbatim)
- Dominant language
- Python
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 23
Description
## Problem statement
The migrator resolves **labels** to OTel semantic conventions (built-in `PROM_TO_OTEL_CANDIDATES` + rule-pack `label_rewrites`/`label_candidates`), but it has **no equivalent mapping for metric names**. Under `--field-profile otel`, `SchemaResolver.resolve_metric_field()` is a pass-through: it returns the source metric name (or `metrics.` only if live caps advertise that exact field). The named Prometheus profiles only add a fixed prefix/suffix, they do not rename.
For any migration that reuses existing OTel data in Elasticsearch (a primary use case), source metric names almost never equal the target field names, so panels translate cleanly and then render empty. Grafana rule packs expose `label_rewrites` / `label_candidates` / `metric_kinds`, none of which rename a metric. This is the Grafana counterpart of Datadog `metric_map` (#307) and aligns with the in-flight `feat-shared-metric-map` work.
## Proposed change (general capability)
Add a metric-name mapping layer, configurable via rule pack, threaded through `resolve_metric_field()` in the same plan→emit→verify model as labels. It must support more than a flat rename:
1. **name → name** rename (`source_metric` → `target_field`).
2. **name → (field, attribute filter)** for attribute-split metrics, where one target metric encodes a dimension the source split across metrics (e.g. separate receive/transmit source metrics collapsing to one target metric plus a `direction` attribute).
3. **per-metric target index / data stream**, since related series may live in different target data streams and a single panel may need more than one source (see #311).
4. **plan→emit→verify parity with labels**: emit the mapped name offline; with `--es-url`, record mapped-vs-missing metrics in `required_target_contract.json`.
5. A shared default map so common conventions (Prometheus / cAdvisor / kube-state-metrics → OTel semconv) work out of the box, extensible per rule pack.
Preserve degrade-gracefully: an unmapped metric with no confirmed target field should surface as a data-readiness gap, not a silently empty panel.
## Source / target impact
Grafana source → OTel Elasticsearch target. Also relevant to any target whose field names differ from source metric names.
## Additional context
Code: `observability_migration/adapters/source/grafana/schema.py` (`resolve_metric_field`, `PROM_TO_OTEL_CANDIDATES`), rule-pack surface in `.../grafana/rules.py`, behavior in `docs/sources/grafana.md`. Requirements (2) and (3) verified against Elastic's managed `kubernetes_otel` dashboards; see comment below for a concrete example and seed mappings. Related: #307, #306, #310, #311, #271, #284.
Contributor guide
Research direction
Start with observability_migration/adapters/source/grafana/schema.py, especially resolve_metric_field() and PROM_TO_OTEL_CANDIDATES, then inspect the rule-pack surface in .../grafana/rules.py and docs/sources/grafana.md. Compare the related metric_map and shared-metric-map work before defining the mapping flow. Done means supporting renames, attribute filters, target data streams, plan→emit→verify reporting, shared defaults, and graceful gaps for unmapped metrics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, grafana, python
- Domain
- backend-api-design, observability-sre, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100