elastic / elastic/observability-migration-platform
Extend translation correctness harness beyond Grafana PromQL (Datadog, alerts, full surface)
- Dominant language
- Python
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 23
Description
## Summary
v1 of the **Translation Correctness Harness** is scoped to **Grafana PromQL → ES|QL** only (design: `docs/superpowers/specs/2026-07-17-grafana-translation-correctness-harness-design.md`).
This issue tracks the deferred surface so we do not pretend Grafana-only coverage means “all translations.”
## Why deferred
Recent production bugs (Node Exporter multi-target merge, join-ratio CASE+bare IRATE, bare vs wrapped `*_OVER_TIME`) were Grafana PromQL emitter-path skew. Fixing that class first has the highest ROI. Datadog and alerts use different IR/emitters and need their own path registry + oracle rules.
## Scope to implement (follow-on)
### 1. Datadog metric / log / formula translations
- Structural oracle adapted to Datadog-emitted ES|QL (`FROM` + agg shapes, not only TS STATS CASE mixes).
- Emitter path matrix for Datadog planner/widget fusion paths (parity with Grafana `GRAFANA_ESQL_EMITTERS`).
- Fixture corpus gate over Datadog sample dashboards / pinned corpus.
- Seed intake from Datadog live_validate / render_error failures.
### 2. Alert rule translations
- Grafana alerting rules + Datadog monitors → Kibana rules.
- Separate oracle: query legality, required fields, action/connector placeholders, enablement safety.
- Must not silently treat `not_feasible` / degraded alerts as success.
### 3. Broader Grafana surface (non-PromQL)
- LogQL / other datasource query types where we claim translation.
- Dashboard variables / controls / links (interaction + render-audit already partial).
- Native PromQL passthrough panels (smoke, not ES\|QL structural rules).
### 4. Optional shared package
- Only after Grafana + Datadog both prove the pattern: extract a shared `translation_oracle` API so sources plug adapters rather than forking checkers.
## Success criteria
- [ ] Datadog PR CI runs structural oracle + emitter matrix (no path skew regressions).
- [ ] Alert translations have an offline gate with disposition (`real_bug` vs data/config gap).
- [ ] `docs/testing.md` documents multi-source harness layout.
- [ ] Seed intake works for at least one non-Grafana source.
## Non-goals
- Claiming perfect semantic parity from offline structure alone (Tier 4 live/compare/render remain authority).
- Blocking the Grafana PromQL v1 harness on this issue.
## References
- Design (v1 Grafana): `docs/superpowers/specs/2026-07-17-grafana-translation-correctness-harness-design.md`
- Pyramid: `docs/superpowers/specs/2026-06-25-migration-confidence-pyramid-design.md`
- Testing narrative: `docs/testing.md`
- Related PR work: #299 (field profiles + multi-target STATS hardenings)
Contributor guide
Assessment
This issue has not been assessed yet.