ADORSYS-GIS / ADORSYS-GIS/lightbridge-authz
[Story]: Cutover: migrate the governance store's rows in, verify counts, decommission
- Lingua principale
- Rust
- Stelle
- 0
- Fork
- 1
- Merge medio
- 6h 42m
- PR unite (30g)
- 246
Descrizione
Parent epic: #581
### Story Statement
As the platform owner, every row the governance store ever collected (Copilot dailies, seat snapshots, executions/model_calls/tool_calls) lives in the usage store with asserted-equal counts, and the governance store's telemetry tables are dropped.
### Real Intent
Execute the cutover ADR-0027/governance ADR-0014 mandate: migrate, verify by counts, decommission. Hard cutover — no dual-write period, no dormant tables.
### Background and Context
Copilot data has a replay path already: governance-ctl archives raw NDJSON to S3 (prefix `copilot-governance/raw/`) and has a `replay` subcommand — replaying through the NEW day-grain ingest API exercises the true write path (the same principle as #528). Execution-grain rows have no raw archive and migrate directly, table-to-table. governance#167's no-loss bar (counts asserted, idempotent reprocessing) applies to both.
### Source of truth (links)
ADR-0027 Decision 1 / Consequences · governance ADR-0014 Decision 3 · governance#167 · governance#181 (the failing verify job — fix or retire as part of cutover)
### Acceptance Criteria
1. Copilot dailies + seats land via S3 replay through the authenticated day-grain ingest path; per-`(day, report)` counts match the governance store's `ingest_manifests` records; mismatches block the cutover loudly.
2. Execution/model-call/tool-call rows migrated with per-table count assertions; `(trace_id, span_id)` dedup means re-running the migration changes nothing.
3. Identity references resolve through `usage_identities` (provider user ids preserved verbatim; no shape validation).
4. After verification, the governance telemetry tables are dropped in the same coordinated change (governance-side PR) — not left dormant; `governance-ctl` is already writing to the usage API by then (governance-side story).
5. Dashboards/readers that pointed at the governance Postgres are repointed or retired in the same window (ai-helm#879/#880 successors noted, not silently broken).
### Out of Scope
Building the ingest paths themselves (prior stories); Grafana dashboard redesign.
### Dependencies and Blockers
Day/seat + execution grain stories; authenticated ingest; governance-side story "governance-ctl writes to the usage ingest API". This story lands LAST in the epic.
### Assumptions
The S3 raw archive is complete for the collected window (verify against `ingest_manifests` before trusting; governance#181's failing verify job is a red flag to resolve first).
### Implementation Notes
Sequence: freeze governance-ctl writes → replay/migrate → count assertions → cut governance-ctl over to the usage API → drop old tables. Keep the freeze window short; day-grain data is daily, so a same-day window is available.
### Test Expectations
Count-assertion harness output committed as evidence; a deliberately-corrupted-row rehearsal proving the mismatch path blocks (sabotage-first).
### Verification evidence
To be attached on the implementing PR: per-table before/after counts, replay logs.
### Human accountable owner
Stephane Segning Lambou
### AI Usage
AI-assisted (Claude): research and drafting. Reviewed and owned by Stephane Segning Lambou.
### Design addendum (2026-08-31): the historical backfill exception, pinned
Going forward, the replay/archive story is the collector's raw OTLP object-storage leg (#589), and per-tool archive code retires. **The one hard exception is this story's historical Copilot data: it never transited the collector, so neither the Grafana stack nor the new archive ever saw it.** governance-ctl's existing S3 NDJSON archive + `replay` is the ONLY source for that history and is used here exactly once, as the migration input (AC 1). After this cutover completes and counts are asserted, governance-ctl's bespoke archive path retires with the rest of its direct-Postgres sink — retirement is sequenced **after** this story, never before.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.