refactor(chat): add shared durable turn-status/result record (N8)

## Current status (updated Jun 26, 2026)

This is **still relevant**, but the implementation plan has changed since the original filing.

The exact `cf_chat_turns` / `_recordTurnStatus` design has **not landed** on `main`. The remaining problem is still that several recovery paths answer **"did this turn finish, and what was its result?"** by looking at different durable artifacts and heuristics instead of one authoritative per-turn record.

What changed since this issue was opened:

- #1690 landed a durable **last terminal replay** record (`recordChatTerminal` / `pendingChatTerminal`, keyed by `CHAT_LAST_TERMINAL_KEY`) so a client that reconnects after recovery exhaustion or a terminal stream error sees the terminal frame over the resume handshake. That solves the reconnect-UX slice of this issue, but it is intentionally only the **latest terminal error/interruption**, not a per-turn status/result ledger.
- #1640 and #1641 substantially improved the sub-agent deploy-churn path: stable child run IDs, bounded reattach, repairable `interrupted`, and parent recovery progress credit from forwarded child chunks. That reduced the urgency of the agent-tool side, but did not remove the heuristic reconcile path.
- #1642 is now complete, so the old plan to add the table separately in `@cloudflare/think` and `@cloudflare/ai-chat` and hoist it later is stale. If we do this now, the durable turn-status primitive should start in the shared `agents/chat` layer and be consumed by both packages.

So: **keep this issue open**, but treat it as a follow-up hardening/refactor, not as the same release-blocking deploy-churn item it was when filed.

## Why this still matters

There are still multiple partially-overlapping sources of truth for turn state:

- **`cf_think_submissions`** is the public durable-submission projection. Startup recovery still has to decide what to do with `running` submission rows in `_recoverSubmissionsOnStart`. This is the remaining N7-shaped risk: a startup sweep can mark a row terminal while chat recovery is still legitimately continuing the same root turn, causing the continuation to later see `submission_not_running` and skip itself.
- **Chat recovery incidents** describe recovery progress and scheduling, but they are not a durable final answer for every completed turn. They are useful operational state, not a stable per-turn outcome ledger.
- **Agent-tool child run rows** still reconcile stale child state by consulting recovery state and transcript-derived terminal evidence. #1640/#1641 made this much more robust, but the model is still "derive a result from surrounding artifacts" rather than "read the child's own turn result".
- **`CHAT_LAST_TERMINAL_KEY`** from #1690 is deliberately narrow: it stores the most recent terminal error/interruption for reconnect replay. It is not retained per request, does not record successful completions, and should not become a general-purpose status ledger.

The remaining value is correctness under rare churn, simpler reasoning, and better diagnostics: given a stable `recoveryRootRequestId`, the framework should be able to answer **running / completed / interrupted / error / aborted / skipped** without re-deriving it from submissions, incidents, stream buffers, and transcript shape.

## Proposed record

Add a shared durable turn-status/result record in `agents/chat`, consumed by `@cloudflare/think` and `@cloudflare/ai-chat`.

Suggested SQLite shape, subject to final naming in the shared layer:

```sql
CREATE TABLE IF NOT EXISTS cf_chat_turns (
request_id TEXT PRIMARY KEY, -- recoveryRootRequestId: stable across continuations
status TEXT NOT NULL, -- running | completed | interrupted | error | aborted | skipped
stream_id TEXT,
result_text TEXT, -- final assistant text / summary when available
error_message TEXT,
reason TEXT, -- recovery / terminal reason, e.g. stable_timeout
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
completed_at INTEGER
)
```

Key points:

- Key by the stable `recoveryRootRequestId`, not a per-continuation request id.
- One row per logical turn; continuations update the same row.
- Terminal statuses are idempotent and should not be overwritten by later non-terminal writes.
- Existing public status columns stay. Submission rows and agent-tool rows remain API-facing projections/caches; their recovery decisions defer to this record when present.
- Missing row means "pre-upgrade or unsupported path" and must fall back to current behavior until coverage is proven universal.

The shared helper should probably live in `packages/agents/src/chat/` next to the recovery incident / resume-handshake primitives. Strawman API:

```ts
recordChatTurnStatus(storage, requestId, status, options)
getChatTurn(storage, requestId)
sweepChatTurns(storage, options)
```

This should be separate from #1690's `recordChatTerminal`: terminal replay is a UX hydration primitive for the latest terminal error; turn status is the authoritative per-request recovery/result primitive.

## Writer coverage

Phase 1 should be write-only and assertive. Every entry and terminal path should write the record, but no behavior should depend on it yet.

Likely write sites:

- Turn accepted / start: normal WebSocket chat, server-side `saveMessages` / continuation, durable submission claim, and agent-tool child turn start.
- Normal completion: the shared/host completion path that already produces `ChatResponseResult` / `onChatResponse` data.
- Normal error / abort: stream `error`, app/provider terminal error, aborted turns, and reset/cancel paths that intentionally terminalize a turn.
- Recovery terminal: recovery exhaustion, incident `completed` / `skipped` / `failed`, and any explicit recovered-submission completion / interruption path.
- Clearing / superseding should not delete per-turn terminal records immediately; use retention/TTL. Unlike `CHAT_LAST_TERMINAL_KEY`, this is history, not just "latest thing to replay".

## Reader coverage

Only after write coverage is proven should readers use the record.

- **N7 / submissions:** `_recoverSubmissionsOnStart` should read the row for `request_id`.
- `completed` -> complete the submission projection.
- `running` -> defer; do not prematurely mark the submission `error` while recovery may still continue.
- `interrupted` / `error` / `aborted` / `skipped` -> project that terminal status onto the submission row.
- missing row -> current heuristic fallback.
- **N6 / agent-tool child reconcile:** stale child-run reconciliation should read the child's turn row first.
- `completed` -> collect the real result / mark completed.
- `running` -> keep waiting or bounded-reattach according to current policy.
- terminal error/interruption -> project terminal result according to existing semantics.
- missing row -> current incident/transcript fallback.
- **Observability / debugging:** expose internal lookup helpers for tests and diagnostics, not necessarily public API.

## Phased rollout

Each phase should be independently shippable.

- [ ] **Phase 0: rebase design on shared chat layer.** Confirm current `agents/chat` recovery extraction points and decide exact module/API names. Do not duplicate the table in both packages.
- [ ] **Phase 1: shared record + write-only integration.** Add table/helper/tests in `agents/chat`, wire all think + ai-chat write sites, and assert the record matches real outcomes. No reader behavior change.
- [ ] **Phase 2: N7 submission recovery reader.** Make `_recoverSubmissionsOnStart` defer to the turn record when present, with current heuristic fallback when absent. Add a regression test where a `submitMessages` turn interrupted mid-loop must not be marked `error` while its root recovery is still `running`.
- [ ] **Phase 3: N6 agent-tool reconcile reader.** Use the child turn record before incident/transcript derivation in think + ai-chat child-run reconcile. Keep fallbacks until we have upgrade confidence.
- [ ] **Phase 4: cleanup.** Once deployed long enough that rows are universal for new turns, remove dead fallback branches where safe and keep only upgrade/backcompat fallbacks that are still needed.

## Impact

Expected positive impact:

- Fewer false terminal states during deploy churn / DO eviction, especially around durable submissions plus chat recovery.
- Simpler agent-tool reconciliation: child run status can be projected from the child's own turn result instead of reconstructed from transcript + incident state.
- Better operational debugging: one durable row answers what happened to a turn by `requestId`.
- Lower future maintenance cost: submission recovery, agent-tool recovery, reconnect terminal UX, and recovery incidents can remain separate concerns instead of overlapping sources of truth.

Expected risk / cost:

- This touches fragile recovery paths where e2e has caught regressions that unit tests missed (#1640). Keep reader changes separate from writer changes.
- Missing a writer can strand a turn as `running`. Phase 1 tests should focus on write coverage and terminal idempotency.
- Storage writes increase slightly. Keep the schema small, update idempotently, and sweep terminal rows on a TTL cadence.
- Reader fallback is mandatory for in-flight turns created before the upgrade.

## Testing gates

Minimum gates for each behavior-changing phase:

- Shared `agents/chat` unit tests for the record helper, terminal idempotency, non-terminal update behavior, and TTL sweep.
- Full `@cloudflare/think` and `@cloudflare/ai-chat` unit suites.
- Real Durable Object / WebSocket recovery tests where applicable.
- SIGKILL / eviction e2e for recovered turns.
- `examples/deploy-churn`, including `--mode subagent`.
- For shared-layer edits, run `nx run agents:build` before think/ai-chat typecheck because sibling packages consume built `agents` output.

## Recommendation

Do this if we are continuing to harden recovery before a broad release, or if we see any more submission/recovery false terminal reports. Do not block a near-term release on the whole multi-phase refactor if the current customer path is already validated by #1640/#1641/#1690.

The safest next PR is **Phase 1 only**: shared record, write-only integration, and tests that prove it mirrors current outcomes. That creates observability and de-risks the later N7/N6 reader changes without changing runtime decisions yet.

## Related

- #1640 — sub-agent stable run id, bounded reattach, transcript-based stale child reconcile.
- #1641 — count forwarded child stream progress as parent recovery progress.
- #1690 — durable latest-terminal replay over resume handshake; solves reconnect frozen-turn UX but is intentionally not a per-turn status ledger.
- #1642 — shared chat recovery/repair layer is now complete; this issue should build on that shared layer rather than duplicating think/ai-chat first.
- Original N7/N8 concern: durable submissions can prematurely terminalize a still-recovering root turn; a unified turn record fixes that by construction once readers defer to it.