Graph as the core artifact: per-correlation sealed recordings, aligned scoring, parallel driving — delivery plan

## Why

A replay produces two execution graphs, and today the system constructs both and consults neither: the verdict classifies over flat call streams (`divergence/mod.rs:2295` — "the verdict below is unaffected"), the dashboard rebuilds a tree from ledger span-path strings because a name-keyed merge of the raw graph forests pairs nothing (`web/src/lib/spine.ts` header), every run re-derives per-correlation structure from the raw tape, a run pulls the entire session (361 MB) to drive 3 correlations, and the kernel drives sequentially in record order although correlations are independent test cases by construction.

#60 (`docs/design/graph-aligned-scorer.md`) defines the comparison half: classification as alignment of two activation forests. This issue is the construction, storage, and delivery half, plus the full plan to land both without regressing the running system. The claim:

**The activation forest is the recording's core artifact.** The lowest unit of storage, selection, replay, and scoring is a single correlation: compaction materializes one forest per correlation with its boundary events attached, a run pulls exactly the correlations it drives, the kernel drives them in parallel in any order, and the scorer aligns forest against forest.

## Target design

1. **Graph as core artifact.** Compaction materializes per-correlation activation forests — graph nodes, boundary events attached by `(recording_run_id, graph_node_id)`, a named unattached annex, per-node subtree-event-count rollups — with balanced accounting asserted: every event attached or in the annex, every node in exactly one correlation forest or the ambient forest.
2. **Per-correlation storage.** `sessions/v1/{id}/correlations/{corr}/{meta.json, tape.ndjsonl.zst, forest.json}`, an `ambient/` sibling (uncorrelated traffic, always in scope), an `index/` of concatenated meta heads for one-GET listing, and a manifest that accretes correlation entries with per-correlation content hashes, written last. Meta heads carry the request vocabulary already captured on root nodes (`EXECUTION_GRAPH_SELECTED_FIELDS`: flow, connector, payment method, merchant, tenant, status) plus counts, gseq range, completeness, and seedability — the picker filters on metadata instead of matching ids.
3. **Additive, early compaction.** A tailing compactor seals correlations as they complete (ingress finalizer + quiesce window), while the session is still recording. Late arrivals reseal that one correlation loudly (`meta.completeness`); correlations overlapping drop-gaps carry `possible_gap: true`.
4. **Runtime substrate first.** Graph nodes state their task lineage; the tape's two sequence spaces are named and drop accounting becomes kind-aware; the record-only `deja::http_incoming` ingress span asymmetry — a live bug demoting rank-2 lookups on every replay today — is fixed at the producer; replay-side nodes carry run identity.
5. **Aligned scoring (#60 implemented).** `detect()` switches per correlation: graph-aligned when both forests exist, the span-scoped flat scorer otherwise, with `scoring_mode` declared per correlation. The call ledger regenerates from the alignment, retiring the pre-#57 pairing identity and the "may be a mis-pairing" UI caveat.
6. **Parallel driving.** Bounded concurrency across correlations, record order within one; `KERNEL_PARALLELISM` from the environment, default 1. The standing invariant is order-invariance: the same scope at K=1 recorded order and K=8 shuffled must produce identical verdicts and per-correlation outcomes.
7. **Explicitly out of scope.** Serving identity stays the lookup ladder; the landing tier stays flat; no cross-correlation alignment; no old-tape backfill (old tapes take the declared flat tier; re-record rather than migrate).

## The tracks

Sizes are S/M/L. Stacks go up with `gh stack`. Every PR leaves the system green on its own; a PR whose behavior flips a default sits behind an environment gate with a declared fallback.

### Track A — runtime substrate

- [ ] **A1** (M) `feat(graph): stamp task lineage onto execution-graph nodes at close` — every tape-carried node states its own `bucket_id`/`fork_seq`, read at span *close* so layer registration order cannot race it; v8→v9, serde-additive. *After G2 pins the v8 fixture.*
- [ ] **A2** (M) `fix(writer): name the tape's sequence spaces and make drop accounting kind-aware` — the boundary and graph counters stop sharing one drop ledger; markers gain per-space truth **additively** (`ranges` stays boundary-only and byte-compatible; `graph_ranges` is a new key), closing the doc/code contradiction in the compatible direction. *After G2.*
- [ ] **A3** (M) `fix(ingress): replay drives requests under the same deja::http_incoming span as record` — kills the systematic rank-2→rank-3 demotion and gives the replay forest the same ingress root the record forest has. Vendor-fork commit. Gate includes a before/after `resolved_by_rank` diff on the same tape, and a check that the oldest tape any environment still replays was recorded with the prefix (pre-prefix tapes are re-record-only, stated in the PR body).
- [ ] **A4** (S) `fix(replay): stamp run identity onto replay-side graph nodes` — symmetric attach identity for the forest builder; consumed defensively (see C5).
- [ ] **A5** (S) **vendor pin bump** — the explicit deployment step that activates A1/A2/A4 in candidate images. Owned here, gated on `demo/run-self-check.sh` 9/9 and the A3 rank histogram check. Without this item the plan can be fully green in CI while real graph-tier runs misalign — nothing else in any track owns it.

### Track B — shared forest builder + per-correlation compaction

- [ ] **B1** (M) `refactor(forest): extract activation-forest resolution primitives into a new deja-forest crate` — parent map, root walking with dangling-parent promotion and the cycle guard, anchor closure, capture-vs-delivery gap classification; one shared home for compactor and orchestrator, without weakening the `ScopedRecording`/`TapeSlot` seam (the `scope_invariant` test survives).
- [ ] **B2** (M) `feat(forest): shared builder — (nodes, events) → per-correlation forests, annex, rollups, balanced accounting` — the pure function both sides call; construction fails on imbalance. The golden fixture pins the join contract: on a single-run stream, attach keys on `graph_node_id` alone, with run identity as a loud assertion rather than a join column (so the replay side works both before and after A5).
- [ ] **B3** (L) `feat(compactor): per-correlation sealed layout (manifest v2)` — correlation objects, meta heads, index, manifest last; v1 sessions stay readable as the declared flat tier. Ports G4's tier-equivalence test to the v2 path as part of the deletion of `seal_session`. Decides index shape (additive fields on `CorrelationSummary` rows) before F4's type mirror lands.
- [ ] **B4** (L) `feat(compactor): tailing incremental compaction` — finalizer + quiesce, late-arrival reseal, `possible_gap`, conditional-PUT manifest accretion. Pins the demo MinIO image tag; the startup self-test distinguishes "store ignores preconditions" (refuse) from "client never sent one" (config bug).
- [ ] **B5** (M) `feat(s3): read the per-correlation sealed form` — the v2 pull path plus the readers Track C needs. The in-process seal fallback goes behind `DEJA_SEAL_LAYOUT` (default v1) so B5 does not silently become the fleet's first v2 writer; an unknown `manifest_version` degrades to a **declared** landing-rescan fallback, never a hard parse error — otherwise a v2-sealed recording is bricked for every pre-B5 reader and rollback is one-way. The one-way-rollback property is stated in the PR body.
- [ ] **B6** (M) `ops(compactor): compaction job image, single-writer contract, IAM` — the tailer finally runs somewhere with the write permission the replay job was deliberately denied in #32 (`s3:PutObject` on `sessions/v1/*`); demo overlay runs it locally.

**Runtime shape: an incremental job over the S3 landing tier — deliberately not a Kafka consumer.** Vector stays the only Kafka consumer; the landing objects are the compactor's input queue and the S3 list is its poll. Three properties force this:
1. *Landing is the accounting boundary.* Ingest accounting requires an input that can be enumerated and re-read so every line becomes an output or a named drop; S3 objects are re-readable forever, Kafka offsets age out with retention.
2. *Seals are content-addressed and must be reproducible.* `seal_id` hashes the raw input lines — re-sealing after a compactor fix only works if the input still exists byte-for-byte.
3. *A second consumer group is a second truth.* Vector-with-acks defines what was durably recorded; a Kafka-consuming compactor could seal lines that never landed (or miss ones that did), diverging from the landing tier every fallback path rescans — the producer/consumer split this codebase's rules exist to kill.

The latency argument for Kafka is illusory: Vector batches to S3 on its own cadence, and the quiesce window for late-closing spans dominates end-to-end freshness anyway; the target is "replayable minutes after recorded," which polling meets. Deployment: a Kubernetes CronJob (1–2 min, `concurrencyPolicy: Forbid` as belt-and-braces — the conditional-PUT manifest write is the real single-writer guard) with its own service account; the compose demo runs the same binary as a looping service since compose has no cron; the existing CLI batch mode remains the degenerate one-pass case for ended sessions. Each pass is a pure function of S3 state (no offsets, no local state), so a crash before the manifest accretion leaves only idempotently-rewritable orphan objects, and promoting to a long-running single-replica tailer later is a deployment change, not a code change.

### Track C — orchestrator consumption

- [ ] **C1** (S) `fix(lifecycle): an empty record graph writes the note the scorecard reads, not stderr` — standalone, lands immediately.
- [ ] **C2** (M) `refactor(lifecycle): choose the run's correlations from the sealed index, tape scan as the declared fallback` — one GET instead of a full tape scan; precondition for C3.
- [ ] **C3** (L) `feat(orchestrator): pull only the scoped correlation objects + ambient` — deps corrected to **B3 + B5 + C2**. Scoped pulls materialize into **scope-keyed slots** (`recordings/{id}/scoped/{scope_hash}/…`), never the shared canonical `events.jsonl` — a scoped tape in the shared slot is silently mistaken for the whole session by older readers and turns the today-benign concurrent re-pull race into a corrupting one. `TapeSlot` owns the new path so the seam grep still holds; the coverage sidecar is an attestation, not the guard.
- [ ] **C4** (M) `feat(orchestrator): ScopedRecording::forests() — the record graph becomes a slice of the seal` — `extract_record_graph` consumes the sealed slice; on-the-fly extraction stays as the declared flat tier. Owns migration 0007 (both `record_forest` and `replay_forest` artifact kinds) so C5 cannot collide on the migration number. Gate includes the demo smoke (lifecycle change), not `just verify` alone.
- [ ] **C5** (M) `feat(lifecycle): materialize the replay-side forest from observed.jsonl through the shared builder` — same code path as the record side; publishes the aligner's second input. Join per the B2 contract (node-id-only, run identity asserted), so it is correct during the A5 pin-lag window instead of annexing every event fleet-wide.
- [ ] ~~C6~~ — folded into F6, which owns the `/graph` response schema; C-track only produces and hydrates the artifacts.

### Track D — the graph-aligned scorer (#60)

- [ ] **D1** (L) `feat(scorer): pure graph aligner over activation forests, with golden alignment tests` — skeleton pruning via stored rollups, k-th same-named-child alignment, `IdentitySkew` reconciliation through `source_event_global_sequence`, suspected-rename shape flag, asserted balanced accounting. Goldens built from the shared synthetic generator (G0): wall correlation, clean correlation, request-only 400, the same-named loop where serving FIFO and alignment k disagree (asserts the skew is reported, not scored).
- [ ] **D2** (S) `feat(scorer): load both execution graphs into RunArtifacts` — record graph from the run's scoped artifact, replay graph split out of `observed.jsonl` in the same pass that yields observed calls.
- [ ] **D3** (L) `feat(scorer): per-correlation graph-aligned scoring tier with declared scoring_mode` — deps gain **E4** (the order-invariance test must precede it) and a defensive rule: a one-sided ingress root demotes that correlation to flat with reason `ingress_root_asymmetry`, so D3 is correct even before A3/A5 reach the candidate image. All existing counters kept as projections; `counter_disagreements()` still balances. Gate: demo smoke with `E2E_EXPECT_SCORING_MODE=graph`.
- [ ] **D4** (M) `feat(ledger): regenerate the call ledger from the alignment` — retires the pre-#57 `(correlation, boundary, method)` pairing and the UnifiedView caveat it necessitated.
- [ ] **D5** (M) `feat(scorer): port rules A/B and race evidence onto graph facts` — verdict-affecting: gate includes the orchestrator smoke, not `just verify` alone.

### Track E — kernel parallel driving

- [ ] **E1** (S) `feat(kernel): declared drive-order contract` — recorded default, seeded shuffle for adversarial runs; boot-env docs tell the truth.
- [ ] **E2** (M) `test(kernel): concurrent-drive integration harness` — K=1 recorded vs K=8 shuffled yield identical per-correlation diff sets, in bare `cargo test`. The fixture **must include null-correlation side-effect calls interleaved across correlations**: uncorrelated traffic shares one global occurrence bucket, and the kernel's own comment says record order matters for it. The test adjudicates that case explicitly — either serving is fixed (shared-bucket occurrences consumed in `source_event_global_sequence` order) or the honest contract is pinned: K>1 refuses or loudly warns on tapes with uncorrelated side-effect traffic. K=1 stays the default until the K=2 smoke passes on a demo tape that demonstrably carries ambient traffic.
- [ ] **E3** (S) `fix(runtime): FileObservedSink counts every dropped line and fails flush loudly` — the `let _ = write_all(...)` swallow is a silent drop that would surface as fabricated omissions; fixed and pinned line-atomic under concurrent writers. Owns the sink test (absorbs the overlapping safety-net item).
- [ ] **E4** (M) `test(orchestrator): scorer order-invariance golden` — the single `order_invariance.rs`; `detect()` invariant to cross-correlation interleavings, with the within-correlation-order negative control. Merges before any D PR and before E2/E5.
- [ ] **E5** (M) `feat(orchestrator): kernel parallelism and drive order are run parameters` — resolved at creation, passed explicitly, stamped in `run.params`; re-POSTing a row re-runs identically.

### Track F — API + dashboard

- [ ] **F1** (S) `ci(web): vitest harness + first spine characterization tests` — stacked on G1's web job (single owner of the first web CI job).
- [ ] **F2** (S) `fix(dashboard): mirror HttpDiff.transport_error and the missing scorecard counters` — live gap: the UI heuristically sniffs status 0 today.
- [ ] **F3** (M) `fix(dashboard): demoted and inconclusive ledger kinds render as what they are, never as Matched` — live bug: `markOf()` returns null for demoted kinds and the detail pane falls through to "values agree".
- [ ] **F4** (S) `fix(dashboard): the correlations client parses CorrelationSummary rows, not strings` — live bug since #28; lands immediately, after B3's index-shape decision is confirmed in writing so the row type is written once.
- [ ] **F5** (M) `feat(replay): correlations endpoint serves meta heads; picker filters by flow/connector/status` — deps: **B3 + B5 + F4**.
- [ ] **F6** (L) `feat(replay): /runs/{id}/graph serves per-correlation forests + ambient, flat tier declared per correlation` — deps: **C4 + C5**. Owns the response schema.
- [ ] **F7** (M) `feat(replay): /graph carries alignment classifications and scoring_mode, read from the persisted artifact, never recomputed` — deps: **D3 (+D4)**.
- [ ] **F8** (L) `feat(dashboard): the spine consumes forests and alignment; the ledger-string tree becomes the declared flat tier`.
- [ ] **F9** (M) `feat(dashboard): the report leads with subtree findings and names its scoring mode` — the design doc's "two facts, not two thousand" claim, made real in the UI.
- [ ] **F10** (S) `feat(dashboard): show a run's kernel parallelism from run.params` — deps: **E5**.

### Track G — safety net, CI, gates

- [ ] **G0** (M) `test-support: shared synthetic tape/forest generator` — **new, land first among test infra**: a dev-only crate generating tagged-NDJSON tapes and node/event sets with forks, same-named sibling loops, seeded interleavings, drop markers, and late-closing spans. Without it, five tracks each build private fixture builders that go stale independently when A1/A2 change the wire — the exact producer/consumer split this codebase's rules exist to kill. Run-0810-class evidence lives in sandbox S3; synthetic shapes are the only goldens hosted CI can run. Consumed by D1, E2, E4, G2, B2.
- [ ] **G1** (S) `ci: web typecheck/build job + just targets` — first web gate; F1 stacks on it.
- [ ] **G2** (M) `test: golden v8 tape, scorecard-v1, and manifest fixtures pin the decode contract` — **must merge before A1/A2** or the fixture pins nothing; generated from G0 with a hand-audited field check, plus one small non-regeneratable demo-sourced line set.
- [ ] **G3** (S) `test: compile the web Scorecard/ledger mirror against the golden fixtures` — one fixture, two enforced copies (a byte-identity check keeps the web copy honest).
- [ ] **G4** (M) `test: sealed fast path and landing rescan read back the same recording` — written against `compact_session` (the surviving path), since B3 deletes `seal_session`; B3 ports it to v2-vs-landing equivalence as part of the deletion diff.
- [ ] ~~G5~~ — dissolved: the sink half is E3's, the scorer-permutation half is E4's. One owner per invariant.
- [ ] **G6** (M) `ci(smoke): the demo compose run becomes the declared end-to-end gate with asserted facts` — `jq`-checked assertions and named extension flags (`E2E_EXPECT_RANK2`, `E2E_EXPECT_CONDITIONAL_PUT`, `E2E_EXPECT_SCORING_MODE`) instead of human-read logs. Pulled forward to land before the B5/C3/D3 merge wave; smoke output pasted in PR bodies from then on. The smoke needs compose + a live Stripe key and cannot run in hosted CI — that is a named property of the gate, not a surprise.
- [ ] **G7** (S) `docs: compatibility matrix, per-track merge gates, rollout/rollback contract` — checked into the repo. One correction already known: the scorecard lives both as the FS artifact and as `replay_runs.scorecard` JSONB read back through the run row, so scorecard additivity must be pinned for both.

## Merge order

Every wave leaves main releasable; the two behavior flips (storage layout, kernel parallelism) sit behind env flags with declared fallbacks.

1. **Wave 0 — freeze before moving:** G1 → F1 → G0 → G2 → G3 → G4 → C1 → F2 → F3 → F4 → E1 → E3 → E4.
2. **Wave 1 — runtime stack:** A2 → A1 → A4 → A3 (vendor) → A5 (pin bump, gated).
3. **Wave 2 — storage stack:** B1 → B2 → B3 → B4 → B5 (flag-gated) → G6 → B6.
4. **Wave 3 — consumption:** C2 → C3 → C4 → C5 → E2 → E5.
5. **Wave 4 — scoring:** D1 → D2 → D3 → D4 → D5.
6. **Wave 5 — presentation:** F5 → F6 → F7 → F8 → F9 → F10 → G7.

## Risk register

- **The unowned deployment step** was the biggest hole in the original breakdown: A1/A2/A4 activate only after the vendor pin bump, and D3/C5/B4 silently depended on it. A5 now owns it, and D3/C5 are written to be correct during the lag window (defensive ingress-root demotion; node-id-only join).
- **Storage cutover is one-way if unguarded.** A v2 manifest written by an ungated fallback bricks the recording for every older reader. `DEJA_SEAL_LAYOUT` default v1 + tolerant unknown-version handling makes the failure mode a slow declared tier instead.
- **Scoped tapes must be structurally invisible to old readers** — scope-keyed slots, not a sidecar contract that only new readers honor. Several sessions share `DEJA_RECORDING_ROOT` on the dev box; the mixed-version hazard is real there, not just in rolling deploys.
- **Parallel driving's stated premise has a known exception**: null-correlation traffic is order-sensitive at K>1. The order-invariance fixture encodes it; the default stays K=1 until the contract is pinned and the K=2 smoke passes on a tape with ambient traffic.
- **The safety net must not be built twice.** Single owners: E3 (sink), E4 (order invariance), G1+F1 (web CI), F6 (`/graph` schema), C4 (migration 0007), G0 (fixtures).

## Open decisions

- Whether serving fixes the shared null-correlation occurrence bucket (consume in `source_event_global_sequence` order) or the K>1 contract excludes uncorrelated-traffic tapes — settled by E2's adjudication.
- Meta-head index shape: additive fields on `CorrelationSummary` rows (current lean) vs a replacement index — B3 decides, in writing, before F4 merges.
- Packing threshold for very large sessions (individual correlation objects vs packed parts with ranged GETs) — ship individual files, state the threshold in the design doc.
- The design doc itself (`docs/design/unified-execution-graph.md`, the construction/storage companion #60's migration step 4 references) distills from this issue and lands with or before Wave 1.