Replication W2: Cursor correctness & divergence detection

**Workstream W2 of #430 · Foundation · the silent-data-loss safety net**

## Status update (2026-07-01)

Design direction 1 (explicit error classification) is now **substantially shipped** as point fixes, and the current-state text below is stale on the details:
- `isPermanentSourceBlobErrorCode` is no longer ENOENT-only — it's `ENOENT || 404 || 500`, with the origin forwarding a structured statusCode (#429 → #443, plus #403 → #405 for the original ENOENT advance-past).
- Core gained a blob-write taxonomy (harper#1480): an aborted blob write stamps a PENDING_TYPE header so reads classify as **503 = transient/re-streamable** vs **500 = confidently permanent**.
- The reconciliation/backfill pass exists: #388 shipped as the proactive blob-repair sweep (`blobRepair.ts`).
- #426 was closed via #428 (cursorless-start full copy) + the leading-dup fast-skip; its **send-side stale-cursor freeze residual is explicitly owned here** (with W4).

What remains is exactly the two items this workstream predicted, and both have live field evidence:
1. **Bounded-retry escalation budget — still missing.** 503 is *deliberately* classified transient with no budget, so a permanently-wedged "transient" source pins the cursor forever. Observed live as the v4→v5 circular-503 wedge (idle-watchdog writes its own PENDING stub → its reads 503 → held forever). The fix shape: count identical-error reconnects per blob, escalate, then advance-and-mark-diverged after a bounded budget — the sweep (#388/`blobRepair.ts`) is the repair loop that makes advance-past safe.
2. **Receive-side sequence-gap detection — still completely absent.** Still the highest-value missing safety net.

## Summary
Convergence rests on a single per-(db,node) resume cursor whose advance is gated by hand-rolled, narrow error classification. It's a *trusted* watermark, not a *checked* one, and it breaks two opposite ways — over-holding (wedge forever) and silent gaps (undetected data loss). There is no receive-side sequence-gap detection, which is the single highest-value missing safety net in replication.

## Root cause / current state *(see status update for what has since shipped)*
- **Over-holding.** Any blob/apply error not on the permanent allow-list holds the cursor, trusting that "a reconnect will re-stream." The allow-list has broadened (ENOENT/404/500), but for an error stuck in the *transient* class (503) the reconnect reproduces the identical error → the cursor is pinned forever. There is no retry budget.
- **Silent gaps.** A follower correctly resumes from `T`, but the leader's per-subscriber send-position diverges and only re-sends the head, so the `[T, head]` window is never re-sent. Nothing detects the hole because the live tail keeps arriving and `connected:true` holds (#426 — observed as ~39% silent data loss).

## Design direction
1. **Total, explicit error classification.** Every blob/apply error carries a stable `.code`; classification is a table with a *safe default plus a bounded-retry budget*. An error that holds the cursor is counted, escalated after N reconnects, and after a bounded number of identical-error reconnects **advances-and-marks-diverged** rather than holding forever. *(Largely shipped — the remaining piece is the bounded-retry budget for the transient class.)*
2. **Receive-side sequence-gap detection.** Track expected vs received sequence ranges so a `[T, head]` hole is *detected and surfaced* (feeds the W8 divergence alarm) instead of masked by live tail traffic.
3. **Reconciliation / backfill pass (#388).** *(Shipped — `blobRepair.ts`.)* Extend it to close the loop behind any new advance-past classes from item 1.

## Scope
- [x] Stable `.code`/statusCode on blob error paths + broadened permanent classification (#443, harper#1480)
- [ ] Bounded-retry budget → escalate → advance-and-mark-diverged for errors stuck in the transient class (the circular-503 wedge)
- [ ] Receive-side expected-vs-received sequence-range tracking (gap detection)
- [ ] Surface gaps/divergence as a signal (W8)
- [x] Reconciliation/backfill pass for records/blobs past the cursor (#388 — `blobRepair.ts`)
- [ ] Send-side stale-cursor freeze (the #426 residual; pairs with W4's per-origin positions)

## Retires / advances
- [x] #426 — closed via #428; the send-side residual is tracked here
- [x] #429 — incomplete source blob misclassified transient (fixed via #443)
- [ ] #385 — interrupted blob persists as corrupt stub, never re-requested
- [ ] #386 — sustained blob timeouts → silent, unrecoverable loss with no signal
- [x] #388 — proactive blob backfill / repair (shipped)
- **Powers:** the divergence alarm in W8

## Dependencies
Pairs naturally with W1 (which provides the surfacing channel) and W8 (which raises the alarm). The send-side residual pairs with W4.

## Effort / risk
**M / medium** *(down from L — classification and backfill shipped; the budget + gap detection remain).*

## Acceptance criteria
- A permanently bad blob advances-and-flags after a bounded number of retries (no eternal wedge), **including one misclassified as transient (503)**.
- A `[T, head]` gap raises a detectable signal rather than silently persisting.
- A diverged record/blob past the cursor is repairable.

---
🤖 Filed by Claude on behalf of Kris.