elastic / elastic/observability-migration-platform

Extend translation correctness harness beyond Grafana PromQL (Datadog, alerts, full surface)

Open
#301 0 comments 0 reactions 1 assignee Claimed by @shmsr View on GitHub
program:trust
Dominant language
Python
Stars
6
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
23

Description

## Summary

v1 of the **Translation Correctness Harness** is scoped to **Grafana PromQL → ES|QL** only (design: `docs/superpowers/specs/2026-07-17-grafana-translation-correctness-harness-design.md`).

This issue tracks the deferred surface so we do not pretend Grafana-only coverage means “all translations.”

## Why deferred

Recent production bugs (Node Exporter multi-target merge, join-ratio CASE+bare IRATE, bare vs wrapped `*_OVER_TIME`) were Grafana PromQL emitter-path skew. Fixing that class first has the highest ROI. Datadog and alerts use different IR/emitters and need their own path registry + oracle rules.

## Scope to implement (follow-on)

### 1. Datadog metric / log / formula translations
- Structural oracle adapted to Datadog-emitted ES|QL (`FROM` + agg shapes, not only TS STATS CASE mixes).
- Emitter path matrix for Datadog planner/widget fusion paths (parity with Grafana `GRAFANA_ESQL_EMITTERS`).
- Fixture corpus gate over Datadog sample dashboards / pinned corpus.
- Seed intake from Datadog live_validate / render_error failures.

### 2. Alert rule translations
- Grafana alerting rules + Datadog monitors → Kibana rules.
- Separate oracle: query legality, required fields, action/connector placeholders, enablement safety.
- Must not silently treat `not_feasible` / degraded alerts as success.

### 3. Broader Grafana surface (non-PromQL)
- LogQL / other datasource query types where we claim translation.
- Dashboard variables / controls / links (interaction + render-audit already partial).
- Native PromQL passthrough panels (smoke, not ES\|QL structural rules).

### 4. Optional shared package
- Only after Grafana + Datadog both prove the pattern: extract a shared `translation_oracle` API so sources plug adapters rather than forking checkers.

## Success criteria
- [ ] Datadog PR CI runs structural oracle + emitter matrix (no path skew regressions).
- [ ] Alert translations have an offline gate with disposition (`real_bug` vs data/config gap).
- [ ] `docs/testing.md` documents multi-source harness layout.
- [ ] Seed intake works for at least one non-Grafana source.

## Non-goals
- Claiming perfect semantic parity from offline structure alone (Tier 4 live/compare/render remain authority).
- Blocking the Grafana PromQL v1 harness on this issue.

## References
- Design (v1 Grafana): `docs/superpowers/specs/2026-07-17-grafana-translation-correctness-harness-design.md`
- Pyramid: `docs/superpowers/specs/2026-06-25-migration-confidence-pyramid-design.md`
- Testing narrative: `docs/testing.md`
- Related PR work: #299 (field profiles + multi-target STATS hardenings)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.