ADORSYS-GIS / ADORSYS-GIS/converse-frontends

[Epic]: A governance dashboards app, on ui-web and chart-core

Open
#327 1 comment 0 reactions 0 assignees View on GitHub
epic
Dominant language
TypeScript
Stars
0
Forks
0
Avg merge
1h 49m
Merged PRs (30d)
253

Description

### Executive Summary

A third application in this monorepo — alongside `apps/console` — presenting AI-usage
governance dashboards: spend, adoption, seat hygiene and agent behaviour across every AI coding
tool in the estate. It shares `packages/ui-web` and `packages/chart-core` with the console but
is a separate app with its own routes, audience and deployment.

It reads `lightbridge-governance`, not the authz usage API. Those are different stores with
different grains (see the boundary decision, `lightbridge-authz`#535).

### Strategic Intent

The governance product's value is the dashboards — "the dashboards are the product", per
`lightbridge-governance`'s own README. Today that database is read by Grafana directly
(their ADR-0003). Grafana is right for operators and wrong for the audience this serves:
engineering leadership asking *"what are we spending per engineer, is anyone using this, and
which seats are idle?"*

Building it here rather than as a fourth Grafana folder means it uses the design system,
auth and i18n the rest of the platform already has.

### Problem Statement

There is no product surface for governance data. The console covers API keys, budgets and
projects — a different audience and a different store. Grafana covers operators. The
buyer-facing questions have no home.

### Desired Outcome

An app whose dashboards answer, on live data, the questions in **Scope** below — with honest
unavailable states rather than fake zeroes, the standard this repo already set in #260 and
#276.

### Scope (In / Out)

**In — dashboards, grouped by the question they answer**

*Spend (request grain — `executions` / `model_calls`, cost from `model_pricing`)*

1. **Total AI spend over time** — by day/week/month, split by provider and by tool.
2. **Spend per engineer** — the headline number; `internalUserId` is authoritative, never
payload `user.email`.
3. **Spend per model** — which models cost what, and the input/output token split behind it.
4. **Cost per execution** — distribution, not just mean; a long tail is the interesting part.
5. **Unknown-cost share** — the fraction of calls with `NULL` cost. A quality gauge on the
pricing table, and it must read as *unknown*, never as zero.

*Adoption and seats (day grain — Copilot tables; later Anthropic/OpenAI/Cursor)*

6. **Active vs engaged users** — the two are different columns and routinely conflated.
7. **Idle seat report** — seats whose `lastActivityAt` predates a window. Directly actionable
spend reduction, and the easiest ROI story in the product.
8. **Seat cost vs utilisation** — `netCostMicroUsd` against interactions, per user.
9. **Team rollup** — via `CopilotUserTeam`. ⚠️ GitHub omits teams with fewer than five seated
users; the panel must say so rather than showing a silently short list.
10. **Repository activity** — coding-agent, code-review and pull-request activity per repo.

*Agent behaviour (request grain — `executions` / `tool_calls`)*

11. **Executions over time**, per tool and per developer.
12. **Most-used tools** — `toolName` frequency; what agents actually do.
13. **Tool latency** — P50/P95 by tool, from `ToolCall.durationMs`.
14. **Execution duration distribution** — where the slow tail is.
15. **Tool-calls per execution** — a proxy for task complexity.

*Cross-cutting*

16. **Tool comparison** — Claude Code vs Codex vs Copilot vs OpenCode, side by side.
⚠️ The single most dangerous panel in the list: Copilot is day-grain aggregates and the
others are request-grain events. It must compare like with like or state plainly that it
cannot.

**Out**

- Operator/SRE dashboards — those stay in Grafana.
- Anything reading the authz usage API — different store, different app (`apps/console`).
- Any panel requiring prompt or completion content. That data is never stored, by decision.
- Writes of any kind. This app is read-only.

### Source of truth (links)

- `lightbridge-governance` RFC-0003 (source taxonomy) and ADR-0013 (ingest invariants) —
`ADORSYS-GIS/lightbridge-governance#157`
- `lightbridge-governance` schema — `crates/governance-core/schema/governance.cstack`
- `lightbridge-authz`#535 — the store-boundary decision this app's data source depends on
- This repo: #298 (console feature set — the sibling app), #260 / #276 (honest unavailable
states), #279 / #285 (Expo retirement — this app is web, and must not depend on
`packages/ui`), #287 (chart/image workflow pattern to copy)

### Stakeholders

Engineering leadership and FinOps as readers; platform team as operators of the source data.

### Key Assumptions

1. `lightbridge-governance` exposes a query API for these panels. **It does not today** — the
dashboards are blocked on it, and that is their long pole, not the UI work.
2. The store boundary resolves such that governance keeps this data (`lightbridge-authz`#535).
If it resolves the other way, this app's data source changes entirely.
3. `ui-web` + `chart-core` are sufficient. `packages/ui` is being deleted (#285) and must not
be used.

### Constraints

- Read-only.
- Honest unavailable states, per #260 / #276 — no fake zeroes, and `NULL` cost renders as
*unknown*, not `0`.
- Never sum across grains. Each panel names its authoritative table.

### Risks

- **Panel 16 is a double-counting trap.** Mixing day-grain aggregates with request-grain events
in one comparison is exactly the defect `lightbridge-authz`'s ingestion audit found (their
F3) — pre-aggregated counters and per-request rows summed together, with the grain filter
defaulting to off.
- **Several panels need signals the ingest path does not capture yet.** The current normalizers
read only `session.id`, `model.name`, `tokens.input`, `tokens.output`, `duration.ms`,
`tool.name` and `user.email`. There is no success/failure status, so an "error rate" panel is
not buildable without ingest work — and Claude Code *does* emit acceptance and
lines-of-code signals that are currently dropped
(`ADORSYS-GIS/lightbridge-governance#96`, `#97`).
- Estimating this before the query API exists will produce a number that is mostly wrong.

### Success metrics

- Every panel renders on live data or states honestly why it cannot.
- No panel sums across grains.
- Idle-seat and spend-per-engineer are answerable without a manual query.

### Child User Stories

To be split once the query API is scoped. The natural cut is the four groups above.

### Human accountable owner

@stephane-segning

### AI Usage Declaration

Drafted by Claude Code (Opus 5). The dashboard list is derived from the columns that actually
exist in `governance.cstack` and from what the current normalizers parse — not from what the
sources could theoretically emit. The gaps in Risks are what that comparison exposed.

### Human verification completed

Not yet — needs review, particularly assumption 1 and the boundary dependency.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.