ADORSYS-GIS / ADORSYS-GIS/lightbridge-governance

[Story]: governance-ctl writes day-grain facts to the authz usage ingest API

Open
#196 2 comments 0 reactions 1 assignee Claimed by @Dericko681 View on GitHub
user-story
Dominant language
Rust
Stars
1
Forks
2
Avg merge
13h 13m
Merged PRs (30d)
110

Description

Tracked by lightbridge-authz epic: ADORSYS-GIS/lightbridge-authz#581

> **Owner ruling, 2026-09-08 (in-session, recorded by the orchestrating session):** governance-ctl writes to OTel directly. The 2026-08-31 addendum is no longer an amendment bolted onto a REST design — it *is* the design, and the body below has been rewritten so there is one story, not a story plus a contradiction. No bespoke day-grain REST endpoint is to be called from this repo.

### Story Statement

As the Copilot connector, I emit day-grain facts and seat snapshots as **OTLP records** through the authenticated edge OTEL collector — not to my own Postgres, and not to a bespoke REST endpoint — and my S3 raw archive + `replay` keep working until the historical cutover retires them.

### Real Intent

Execute ADR-0014 Decision 2 for `governance-ctl`: the collector stays here, the tables move to the authz usage store. Pull/normalize/verify logic is untouched; only the sink changes — and the sink is the same OTLP door every other push source uses, so auth, the trusted-source stamp (`github_copilot`), and the edge fan-out to the governed store and the raw archive are inherited rather than reinvented (ADORSYS-GIS/lightbridge-authz#589). governance-ctl already speaks OTLP for its operational run metrics, so the plumbing exists.

### Background and Context

`governance-ctl` (`sync`/`sync-day`/`replay`/`verify`) currently upserts Copilot org/user/repo/user-team dailies and seat snapshots into the governance Postgres via cratestack (`crates/governance-copilot/src/store.rs`), with `ingest_manifests` idempotency and raw NDJSON archived to S3. Under consolidation (authz ADR-0027 / this repo's ADR-0014) those rows become the usage store's generalized `usage_day_facts` / `usage_seat_snapshots`, fed by the day-grain normalizer on the usage side reading the OTLP records this story emits.

### Source of truth (links)

ADR-0014 (this repo) · lightbridge-authz ADR-0027 · lightbridge-authz#581 (epic) · #583 · #585 · #588 · #589 · owner ruling 2026-09-08

### Acceptance Criteria

1. `sync`/`sync-day` emit day-grain and seat records as **OTLP log records with typed attributes** through the authenticated edge collector. The direct-Postgres telemetry write path is **removed**, not flagged off.
2. The credential is the collector-facing one, decided jointly with lightbridge-authz#585 — ⚠️ **the old "#169's projected-SA-token pattern" citation is void**: #169 was closed `NOT_PLANNED`, the only `automountServiceAccountToken` settings in this repo turn it *off*, and no TokenReview machinery exists. Whichever mechanism #585 lands is the one used here; refusal on a missing or wrong-audience credential is integration-tested.
3. The OTLP encoding is documented: attribute names, one record per (report, subject), units and types pinned, with a worked example per report kind. It is a contract with the usage-side normalizer, not an implementation detail.
4. Idempotency holds end to end: re-emitting a day changes no counts in the usage store (the natural-key upsert on the usage side is the mechanism). `ingest_manifests` bookkeeping either moves server-side or is re-pointed — decide and record which, in this issue, before implementation.
5. `replay` replays the S3 raw NDJSON through the same OTLP path, and a replayed day matches its original counts. This path is retained **only** until the historical cutover (lightbridge-authz#588) completes; afterwards forward-going replay is the collector archive's generic job (#589) and the bespoke NDJSON archive retires.
6. `verify`/`verify-attribution` are re-pointed at the usage store's query API. (#181, the silently-failing 03:30 verify schedule, is **closed COMPLETED** — its fix is no longer part of this story; only the re-point remains.)
7. Operational OTLP gauges (`governance.copilot.*`) keep flowing; `rows{report}` reflects records the collector accepted, and a partial accept is surfaced as an error metric, not swallowed.

### Out of Scope

The usage-side tables, normalizer and ingest door (lightbridge-authz#583/#585/#589); migrating historical rows (lightbridge-authz#588). This story only switches the live sink.

### Dependencies and Blockers

Blocked on **lightbridge-authz#583** (day/seat-grain tables) and **#585** (authenticated multi-source ingest + the credential decision), both OPEN and unstarted; **#589** (raw OTLP archive) gates AC 5's retirement clause. Coordinate the cutover window with #588 so no day is written to both stores.

### Assumptions

A full day of Copilot reports fits comfortably in batched OTLP export (dailies are small; no streaming needed).

### Implementation Notes

Touches `app/governance-ctl/src/sync.rs` (1178 LoC, already over the 200-LoC ratchet — #178); do that split as part of, or immediately before, this change rather than growing the file further.

### Estimate

⚠️ The original 2 points predate both the OTLP addendum and AC 3. Re-estimate before scheduling, or split AC 6 (the `verify` re-point) into its own ticket.

### Test Expectations

Refusal tests fail-first; replay-parity test; a partial-accept surfaced-as-error test; an encoding round-trip test against the documented attribute contract.

### Verification evidence

To be attached on the implementing PR.

### Human accountable owner

Stephane Segning Lambou

### AI Usage

AI-assisted (Claude Opus 5): research, drafting, and this rewrite. The design decision (OTel direct) is the owner's. Reviewed and owned by Stephane Segning Lambou.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.