ADORSYS-GIS / ADORSYS-GIS/lightbridge-authz

[Story]: Cutover: migrate the governance store's rows in, verify counts, decommission

Aperta
#588 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
user-story
Lingua principale
Rust
Stelle
0
Fork
1
Merge medio
6h 42m
PR unite (30g)
246

Descrizione

Parent epic: #581

### Story Statement

As the platform owner, every row the governance store ever collected (Copilot dailies, seat snapshots, executions/model_calls/tool_calls) lives in the usage store with asserted-equal counts, and the governance store's telemetry tables are dropped.

### Real Intent

Execute the cutover ADR-0027/governance ADR-0014 mandate: migrate, verify by counts, decommission. Hard cutover — no dual-write period, no dormant tables.

### Background and Context

Copilot data has a replay path already: governance-ctl archives raw NDJSON to S3 (prefix `copilot-governance/raw/`) and has a `replay` subcommand — replaying through the NEW day-grain ingest API exercises the true write path (the same principle as #528). Execution-grain rows have no raw archive and migrate directly, table-to-table. governance#167's no-loss bar (counts asserted, idempotent reprocessing) applies to both.

### Source of truth (links)

ADR-0027 Decision 1 / Consequences · governance ADR-0014 Decision 3 · governance#167 · governance#181 (the failing verify job — fix or retire as part of cutover)

### Acceptance Criteria

1. Copilot dailies + seats land via S3 replay through the authenticated day-grain ingest path; per-`(day, report)` counts match the governance store's `ingest_manifests` records; mismatches block the cutover loudly.
2. Execution/model-call/tool-call rows migrated with per-table count assertions; `(trace_id, span_id)` dedup means re-running the migration changes nothing.
3. Identity references resolve through `usage_identities` (provider user ids preserved verbatim; no shape validation).
4. After verification, the governance telemetry tables are dropped in the same coordinated change (governance-side PR) — not left dormant; `governance-ctl` is already writing to the usage API by then (governance-side story).
5. Dashboards/readers that pointed at the governance Postgres are repointed or retired in the same window (ai-helm#879/#880 successors noted, not silently broken).

### Out of Scope

Building the ingest paths themselves (prior stories); Grafana dashboard redesign.

### Dependencies and Blockers

Day/seat + execution grain stories; authenticated ingest; governance-side story "governance-ctl writes to the usage ingest API". This story lands LAST in the epic.

### Assumptions

The S3 raw archive is complete for the collected window (verify against `ingest_manifests` before trusting; governance#181's failing verify job is a red flag to resolve first).

### Implementation Notes

Sequence: freeze governance-ctl writes → replay/migrate → count assertions → cut governance-ctl over to the usage API → drop old tables. Keep the freeze window short; day-grain data is daily, so a same-day window is available.

### Test Expectations

Count-assertion harness output committed as evidence; a deliberately-corrupted-row rehearsal proving the mismatch path blocks (sabotage-first).

### Verification evidence

To be attached on the implementing PR: per-table before/after counts, replay logs.

### Human accountable owner

Stephane Segning Lambou

### AI Usage
AI-assisted (Claude): research and drafting. Reviewed and owned by Stephane Segning Lambou.

### Design addendum (2026-08-31): the historical backfill exception, pinned

Going forward, the replay/archive story is the collector's raw OTLP object-storage leg (#589), and per-tool archive code retires. **The one hard exception is this story's historical Copilot data: it never transited the collector, so neither the Grafana stack nor the new archive ever saw it.** governance-ctl's existing S3 NDJSON archive + `replay` is the ONLY source for that history and is used here exactly once, as the migration input (AC 1). After this cutover completes and counts are asserted, governance-ctl's bespoke archive path retires with the rest of its direct-Postgres sink — retirement is sequenced **after** this story, never before.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.