Grafana alert pack: codify the operational watch-list as importable alerts
- Dominant language
- Rust
- Stars
- 32
- Forks
- 5
- Avg merge
- 15h 21m
- Merged PRs (30d)
- 21
Description
## Motivation
The 0.5→0.6 upgrade guide and the managed-Postgres guide define a precise watch-list (ring rotation, prune blockers, queue lag, xact age), but operators must hand-build alerts from prose. `docs/grafana/` ships dashboards; it should ship alert rules too.
## Scope
Importable Prometheus/Grafana alert rules beside `docs/grafana/awa-dashboard.json`, covering at minimum:
- `pg_stat_activity.xact_start` age beyond threshold (the #169 MVCC-discipline alert promised in that issue's playbook)
- `awa_maintenance_rotate_attempts_total{awa_ring_outcome="skipped_busy"}` sustained elevation (producers outpacing consumers)
- `awa_maintenance_prune_attempts_total{awa_ring_outcome="blocked"}` non-zero (held-tx blocking reclaim)
- `awa_queue_lag_seconds` p95 vs SLO
- DLQ depth delta over window
- `awa_maintenance_branch_overrun_total` non-zero (also the telemetry gate for #303)
- backpressure signals once #341 lands
Each rule documented with meaning, likely cause, and the matching `awa doctor` check / troubleshooting entry.
## Acceptance
- Rules import cleanly into a stock Prometheus/Grafana stack; validated in CI (syntax/lint at minimum).
- Cross-referenced from the upgrade guide and operations handbook so prose watch-lists point at the shipped rules.
## References
Roadmap: `docs/0.7-roadmap.md` (WS-4, NEW-12). Related: #169 playbook, #302/#303, #341, `docs/upgrade-0.5-to-0.6.md` watch-list.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with docs/grafana/awa-dashboard.json, the 0.5-to-0.6 upgrade guide, the managed-Postgres guide, and the referenced #169 playbook to map each watch-list item to its metric and meaning. Check how the stock Prometheus/Grafana stack validates imported rules and how CI performs syntax or lint checks. Done means the rules are importable, validated in CI, and cross-referenced from both required guides.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- grafana, postgresql, prometheus
- Domain
- documentation, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100