ADORSYS-GIS / ADORSYS-GIS/ai-helm

[Ticket]: Four Copilot governance dashboards and five ingestion alerts

Aperta
#880 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
governance grafana observability ticket
Lingua principale
Go Template
Stelle
3
Fork
1
Merge medio
19h 24m
PR unite (30g)
80

Descrizione

### Type

Feature

### Summary

We need **four Copilot governance dashboards and five alerts**, because the dashboards are the actual product of the governance work — collecting the data is only the means.

Expected result:
> Four generated dashboards reading the governance Postgres datasource (plus one reading Mimir for collector health), and five alerts, all committed as generated JSON with their Python sources.

### Intent

Decision 4 in the roadmap: **the real product is the Grafana dashboard giving a painful information résumé.** Everything upstream of this ticket exists to make these four views possible.

### Source of truth (links)

- Epic: ADORSYS-GIS/lightbridge-governance#5
- Depends on: ADORSYS-GIS/lightbridge-governance#12 (collector), #879 (datasource)
- Roadmap: ADORSYS-GIS/ai-helm#872 §4 Wave 2, decision 4
- Plan detail: `plans/github-copilot-governance.md` §0 deviation 3, and the dashboards/alerts sections

### Current Behavior

No Copilot data is collected and no dashboard exists. Copilot spend is invisible next to gateway spend, which is measured to the micro-dollar.

### Expected Behavior

Four dashboards:

1. **Usage** — who is using Copilot, on what, over time.
2. **Seats** — allocated vs. active; ⚠️ **idle paid seats** are the one that pays for this work.
3. **Teams** — usage by team, via `user-teams-1-day`.
4. **Collector health** — the only one reading **Mimir**; the other three read Postgres.

Five alerts on ingestion health: no data, stale data, repeated fetch failures, gap not closing, backup/archive failure.

### Acceptance Criteria

- [ ] Given the governance datasource, when dashboards 1–3 load, then they query **Postgres**, with usernames/repos/teams as **columns** — not as Prometheus labels.
- [ ] Given dashboard 4, when it loads, then it reads the ~10 low-cardinality operational metrics from Mimir.
- [ ] Given the seats dashboard, when it loads, then **idle seats are identifiable** — the reason this connector was funded.
- [ ] Given the teams dashboard, when it loads, then it carries a visible caveat that ⚠️ **GitHub omits teams with fewer than 5 seated Copilot users**. Team totals must not be presented as complete.
- [ ] Given Copilot spend and gateway spend, when both are shown, then they are in the **same unit** (integer micro-USD) and are directly comparable.
- [ ] Given the five alerts, when each fires, then it is actionable — it says what broke and what to do.
- [ ] Given the dashboard sources, when they change, then `uv run dashboards build` is run and the **regenerated JSON is committed** — `dashboards-drift` CI fails otherwise.

### Out of Scope

- The collector — separate story.
- The datasource — separate ticket.
- Foundry dashboards.
- Any bespoke UI. Grafana is the surface.

### Technical Context

- Sources in `tools/dashboards/` — **`uv` only**, never pip or poetry; `ruff` for lint/format; SDK is `grafana-foundation-sdk`.
- Dashboards are provisioned via grafana-operator.
- ⚠️ **Grafana here runs stateless** (emptyDir), so every pod roll wipes operator-pushed folders. **Set `resyncPeriod` on every folder** or `folderRef` dashboards fail with `[400] folder not found` after a roll, and stay broken until the operator is restarted. This has bitten before; it is not theoretical.
- ⚠️ If any dashboard is imported by `gnetId` and uses the nested datasource format, the string-form `datasource:` does **not** substitute the token — use the list form, with the datasource **UID** lowercase.

### Risks

| Risk | Probability | Impact | Mitigation |
|---|---|---|---|
| **Team totals read as complete when GitHub omits small teams** | **High** | Medium | Caveat on the panel itself, not only in docs |
| Folders vanish after a Grafana roll | **High** if `resyncPeriod` omitted | Medium | Set it on every folder |
| Generated JSON not committed → CI red | Medium | Low | `dashboards build` before pushing |
| Alerts too noisy and get muted | Medium | High | Alert on "ingestion is broken", not on every transient fetch retry |
| Dashboards built before data exists, and look wrong | Medium | Low | Sequence after the collector, or use seeded data knowingly |

### Test Plan

1. `uv run dashboards build`; confirm no drift (`uv run dashboards check`).
2. `uv run ruff format . && uv run ruff check .`
3. Load each dashboard against real collected data — not seeded, where possible.
4. Force each alert condition and confirm it fires and reads sensibly.
5. Roll the Grafana pod and confirm folders and dashboards survive.

### Verification evidence

*To be completed on delivery.*

Required:
- `uv run dashboards check` clean.
- Screenshots of all four dashboards with real data.
- Evidence that a Grafana pod roll does not break the folders (the `resyncPeriod` check).
- Each alert shown firing at least once.
- A panel showing Copilot and gateway spend in the same unit.

### Human accountable owner

@stephane-segning

### AI Usage Declaration

Drafting the ticket

### Human verification completed

- [x] I understood the intent
- [x] I checked the source of truth
- [x] I documented remaining risks
- [x] I am the accountable owner and accept responsibility for this ticket.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.