elastic / elastic/observability-migration-platform

Grafana alert migration must honor query type (Instant vs Range)

Open
#210 0 comments 0 reactions 0 assignees View on GitHub
asset:alerts enhancement program:grafana-engine source:grafana workstream:stability
Dominant language
Python
Stars
6
Forks
8
Avg merge
2d 22h
Merged PRs (30d)
23

Description

## Product goal

Operators migrating Grafana alerts should not have to manually fix query evaluation mode after migration. A rule configured as **Instant** or **Range** in Grafana should produce a Kibana rule that evaluates metrics the same way — without silent semantic drift.

## User problem

Grafana exposes two query modes in the alert form:

| Grafana UI | Operator intent |
|---|---|
| **Instant** | Evaluate the **current value** (“WHEN QUERY IS ABOVE N”) |
| **Range** | Evaluate over a time window at a chosen **Interval** / resolution |

The migration pipeline currently treats alert queries uniformly and emits a **range-style** query (`step=1m`) for both. That causes:

- **Instant rules over-fire** on stale values that already recovered (#200)
- **Range rules run at the wrong resolution** when Interval ≠ 1m (#209)

Operators see a plausible PromQL expression and threshold, but alert behavior does not match Grafana.

## Desired outcome

| Source configuration | Migrated behavior |
|---|---|
| **Instant** (`instant: true`) | Single evaluation at rule run time → `time=now()`, no `step` |
| **Range** (`range: true`) | Window evaluation at source resolution → derived `step=` from Interval / `intervalMs` |

## Out of scope (separate tracks)

- **Reduce / Math / Classic / multi-step graphs** → #194 (complex expression cases)
- **Pending period (`for`)** → #174
- **Rule schedule interval** → #173 (closed)
- **Reducer semantics** (last / avg / min) → #194

## Delivery breakdown

This epic is satisfied when both child issues are done:

| Issue | Scope |
|---|---|
| #200 | Instant alerts → `time=now()`, no `step`; no over-fire on recovered metrics |
| #209 | Range alerts → `step=` derived from Grafana Interval / metadata, not hardcoded `1m` |

Implementation should read `instant` / `range` from the exported Grafana query model when choosing the PROMQL selector.

## Success criteria (epic level)

- [ ] Instant test alert: metric dips then recovers → migrated rule does **not** fire on stale breach
- [ ] Range test alert with Interval 1s → migrated rule uses `step=1s`, not `step=1m`
- [ ] Migration comparison/report distinguishes Instant vs Range source rules
- [ ] Documentation or skill guidance updated for reviewers: what to check on migrated alert queries

## How to validate (QA)

Use a small Grafana alert fixture set:

1. **Instant** — simple threshold, metric recovers before evaluation → no false alarm after migration (#200)
2. **Range** — Interval 1s, 10m window → emitted step matches source (#209)
3. Regression — neither mode emits the wrong selector for the other mode

## Related

- #200 — Instant query type
- #209 — Range step / resolution
- #194 — Complex expression graph
- #174 — Pending period
- Parent: elastic/observability-migration-platform#71

Contributor guide

Open the contributing guide

Research direction

Start by reading child issues #200 and #209, then trace the migration pipeline’s use of the exported Grafana query model and its instant/range fields. Validate with the small Instant and Range Grafana fixture cases described in the issue; done means selectors preserve query type and Interval resolution, the comparison report distinguishes them, and reviewer guidance is updated.

Written by the indexing model from the issue text.

Assessment

Tech stack
grafana, python
Domain
observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.