ADORSYS-GIS / ADORSYS-GIS/ai-helm

[Ticket]: Four Copilot governance dashboards and five ingestion alerts

Đang mở
#880 2 bình luận 0 reaction 0 người được giao Xem trên GitHub
governance grafana observability ticket
Ngôn ngữ chính
Go Template
Star
3
Fork
1
Merge trung bình
19 giờ 24 phút
Pull request đã merge (30 ngày)
80

Mô tả

### Type

Feature

### Summary

We need **four Copilot governance dashboards and five alerts**, because the dashboards are the actual product of the governance work — collecting the data is only the means.

Expected result:
> Four generated dashboards reading the governance Postgres datasource (plus one reading Mimir for collector health), and five alerts, all committed as generated JSON with their Python sources.

### Intent

Decision 4 in the roadmap: **the real product is the Grafana dashboard giving a painful information résumé.** Everything upstream of this ticket exists to make these four views possible.

### Source of truth (links)

- Epic: ADORSYS-GIS/lightbridge-governance#5
- Depends on: ADORSYS-GIS/lightbridge-governance#12 (collector), #879 (datasource)
- Roadmap: ADORSYS-GIS/ai-helm#872 §4 Wave 2, decision 4
- Plan detail: `plans/github-copilot-governance.md` §0 deviation 3, and the dashboards/alerts sections

### Current Behavior

No Copilot data is collected and no dashboard exists. Copilot spend is invisible next to gateway spend, which is measured to the micro-dollar.

### Expected Behavior

Four dashboards:

1. **Usage** — who is using Copilot, on what, over time.
2. **Seats** — allocated vs. active; ⚠️ **idle paid seats** are the one that pays for this work.
3. **Teams** — usage by team, via `user-teams-1-day`.
4. **Collector health** — the only one reading **Mimir**; the other three read Postgres.

Five alerts on ingestion health: no data, stale data, repeated fetch failures, gap not closing, backup/archive failure.

### Acceptance Criteria

- [ ] Given the governance datasource, when dashboards 1–3 load, then they query **Postgres**, with usernames/repos/teams as **columns** — not as Prometheus labels.
- [ ] Given dashboard 4, when it loads, then it reads the ~10 low-cardinality operational metrics from Mimir.
- [ ] Given the seats dashboard, when it loads, then **idle seats are identifiable** — the reason this connector was funded.
- [ ] Given the teams dashboard, when it loads, then it carries a visible caveat that ⚠️ **GitHub omits teams with fewer than 5 seated Copilot users**. Team totals must not be presented as complete.
- [ ] Given Copilot spend and gateway spend, when both are shown, then they are in the **same unit** (integer micro-USD) and are directly comparable.
- [ ] Given the five alerts, when each fires, then it is actionable — it says what broke and what to do.
- [ ] Given the dashboard sources, when they change, then `uv run dashboards build` is run and the **regenerated JSON is committed** — `dashboards-drift` CI fails otherwise.

### Out of Scope

- The collector — separate story.
- The datasource — separate ticket.
- Foundry dashboards.
- Any bespoke UI. Grafana is the surface.

### Technical Context

- Sources in `tools/dashboards/` — **`uv` only**, never pip or poetry; `ruff` for lint/format; SDK is `grafana-foundation-sdk`.
- Dashboards are provisioned via grafana-operator.
- ⚠️ **Grafana here runs stateless** (emptyDir), so every pod roll wipes operator-pushed folders. **Set `resyncPeriod` on every folder** or `folderRef` dashboards fail with `[400] folder not found` after a roll, and stay broken until the operator is restarted. This has bitten before; it is not theoretical.
- ⚠️ If any dashboard is imported by `gnetId` and uses the nested datasource format, the string-form `datasource:` does **not** substitute the token — use the list form, with the datasource **UID** lowercase.

### Risks

| Risk | Probability | Impact | Mitigation |
|---|---|---|---|
| **Team totals read as complete when GitHub omits small teams** | **High** | Medium | Caveat on the panel itself, not only in docs |
| Folders vanish after a Grafana roll | **High** if `resyncPeriod` omitted | Medium | Set it on every folder |
| Generated JSON not committed → CI red | Medium | Low | `dashboards build` before pushing |
| Alerts too noisy and get muted | Medium | High | Alert on "ingestion is broken", not on every transient fetch retry |
| Dashboards built before data exists, and look wrong | Medium | Low | Sequence after the collector, or use seeded data knowingly |

### Test Plan

1. `uv run dashboards build`; confirm no drift (`uv run dashboards check`).
2. `uv run ruff format . && uv run ruff check .`
3. Load each dashboard against real collected data — not seeded, where possible.
4. Force each alert condition and confirm it fires and reads sensibly.
5. Roll the Grafana pod and confirm folders and dashboards survive.

### Verification evidence

*To be completed on delivery.*

Required:
- `uv run dashboards check` clean.
- Screenshots of all four dashboards with real data.
- Evidence that a Grafana pod roll does not break the folders (the `resyncPeriod` check).
- Each alert shown firing at least once.
- A panel showing Copilot and gateway spend in the same unit.

### Human accountable owner

@stephane-segning

### AI Usage Declaration

Drafting the ticket

### Human verification completed

- [x] I understood the intent
- [x] I checked the source of truth
- [x] I documented remaining risks
- [x] I am the accountable owner and accept responsibility for this ticket.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Start in tools/dashboards/ and read plans/github-copilot-governance.md, along with the collector and datasource dependencies in issues #12 and #879. Build and check the generated dashboards with uv run dashboards build and uv run dashboards check, then run Ruff and validate the four dashboards, five alerts, folder resync behavior, and comparable spend units against the acceptance criteria.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
helm, kubernetes, postgresql, python
Lĩnh vực
data, infrastructure, observability-sre
Loại issue
Tính năng
Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức độ hoạt động
Sôi nổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
38/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.