elastic / elastic/observability-migration-platform

[Grafana] Add a source→OTel metric-name mapping layer (otel profile currently passes metric names through verbatim)

Open
#309 5 comments 0 reactions 0 assignees View on GitHub
asset:dashboards capabilities quality source:grafana workstream:translator
Dominant language
Python
Stars
6
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
23

Description

## Problem statement

The migrator resolves **labels** to OTel semantic conventions (built-in `PROM_TO_OTEL_CANDIDATES` + rule-pack `label_rewrites`/`label_candidates`), but it has **no equivalent mapping for metric names**. Under `--field-profile otel`, `SchemaResolver.resolve_metric_field()` is a pass-through: it returns the source metric name (or `metrics.` only if live caps advertise that exact field). The named Prometheus profiles only add a fixed prefix/suffix, they do not rename.

For any migration that reuses existing OTel data in Elasticsearch (a primary use case), source metric names almost never equal the target field names, so panels translate cleanly and then render empty. Grafana rule packs expose `label_rewrites` / `label_candidates` / `metric_kinds`, none of which rename a metric. This is the Grafana counterpart of Datadog `metric_map` (#307) and aligns with the in-flight `feat-shared-metric-map` work.

## Proposed change (general capability)

Add a metric-name mapping layer, configurable via rule pack, threaded through `resolve_metric_field()` in the same plan→emit→verify model as labels. It must support more than a flat rename:

1. **name → name** rename (`source_metric` → `target_field`).
2. **name → (field, attribute filter)** for attribute-split metrics, where one target metric encodes a dimension the source split across metrics (e.g. separate receive/transmit source metrics collapsing to one target metric plus a `direction` attribute).
3. **per-metric target index / data stream**, since related series may live in different target data streams and a single panel may need more than one source (see #311).
4. **plan→emit→verify parity with labels**: emit the mapped name offline; with `--es-url`, record mapped-vs-missing metrics in `required_target_contract.json`.
5. A shared default map so common conventions (Prometheus / cAdvisor / kube-state-metrics → OTel semconv) work out of the box, extensible per rule pack.

Preserve degrade-gracefully: an unmapped metric with no confirmed target field should surface as a data-readiness gap, not a silently empty panel.

## Source / target impact

Grafana source → OTel Elasticsearch target. Also relevant to any target whose field names differ from source metric names.

## Additional context

Code: `observability_migration/adapters/source/grafana/schema.py` (`resolve_metric_field`, `PROM_TO_OTEL_CANDIDATES`), rule-pack surface in `.../grafana/rules.py`, behavior in `docs/sources/grafana.md`. Requirements (2) and (3) verified against Elastic's managed `kubernetes_otel` dashboards; see comment below for a concrete example and seed mappings. Related: #307, #306, #310, #311, #271, #284.

Contributor guide

Open the contributing guide

Research direction

Start with observability_migration/adapters/source/grafana/schema.py, especially resolve_metric_field() and PROM_TO_OTEL_CANDIDATES, then inspect the rule-pack surface in .../grafana/rules.py and docs/sources/grafana.md. Compare the related metric_map and shared-metric-map work before defining the mapping flow. Done means supporting renames, attribute filters, target data streams, plan→emit→verify reporting, shared defaults, and graceful gaps for unmapped metrics.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, grafana, python
Domain
backend-api-design, observability-sre, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.