writeSchedule lowers only the writing node's claim floor, but rows are residency-routed to their owner — ~75% of writes can strand rows silently

> **Rewritten 2026-08-19.** The first version of this issue blamed *replication*. That was wrong —
> `RenderSchedule` is not replicated; rows are residency-routed and each key lives on exactly one
> node (verified: a 27-key probe returned 1/16/10/0 across four nodes, perfectly disjoint). The
> actual mechanism is below, and it is worse, because it fires from ordinary in-plugin writes.

## Summary

`writeSchedule` lowers the claim floor of the node **executing the write**. But the row it just
wrote is residency-routed to whichever node **owns** that cacheKey. On a 4-node cluster ~75% of
writes therefore lower a floor belonging to the wrong node, and the owner's floor is never lowered
at all.

Any row filed with a due minute older than `queue.claimFloor.guard` (default 5 min) lands **below
its owner's floor** and is never claimed again — silently, permanently, from a fully funnel-routed
write.

## The mechanism

```js
export const writeSchedule = async (cacheKey, { nextRenderTime, fromSitemap } = {}) => {
...
await scheduleTable().put(cacheKey, { nextRenderTime, fromSitemap });
lowerFloorFor(nextRenderTime); // ← local SharedBuffer, i.e. THIS node's floor
};
```

`lowerFloorFor` → `leaseTable().lowerFloorTo(...)` is an `Atomics` CAS-min against the local
`coordination.SharedBuffer` (`replicate: false`, node-local by construction). The `put`, however,
is residency-routed — and per the module comment on `writeSchedule`, deliberately does not block
on the owner:

> a write to a residency-pinned key this node does not own does NOT block on the owner (measured:
> 500 writes in 10.7 ms, mean 0.021 ms, against residency pinned to a node that does not exist)

Ownership is `getResidencyByUrl` → `fnv1a32(\`${url}|${node}\`)` rendezvous hashing, so for any
given writer 3 of 4 keys are owned elsewhere.

## This is already documented in the tree

`resources/Target.js`, in `revalidate`'s phase 2 — the hazard is known and currently mitigated
only by writers remembering to recompute the minute per URL:

> THE CURRENT MINUTE, PER URL — never captured once for the whole sweep. Phase 2 writes up to
> `scan.collectCap` × devices rows with a `PrerenderedPage.get` per key, which at scale takes tens
> of minutes. Rows are residency-routed, so ~75% land on nodes whose claim floor this process
> cannot lower and which hold it at `nowMinute − queue.claimFloor.guard`: every row filed with a
> minute more than the guard band old lands BELOW the owner's floor and is never claimed again —
> silently, from a fully funnel-routed in-plugin write, and permanently where `resetInterval: 0`.
> `Sitemap.js` already computes it per entry for the same reason.

So the invariant "never let more than `guard` elapse between computing a due minute and writing
it" is load-bearing across every schedule writer, enforced nowhere, and unobservable when broken.
Two call sites currently carry hand-written comments to remember it.

## Impact

Observed on a 4-node production cluster (plugin 0.49.0 / harper-pro 5.2.3). One node accumulated
**92,873 rows below its floor across 2,101 distinct due-minutes**, oldest due 2026-08-01 — 18 days
of a growing slice of its shard silently not rendering, while the node reported healthy and kept
serving. Peer nodes were at 507 / 3 / 1 overdue.

Downstream, those keys held damaged blobs that could only be replaced by a re-render, so
replication base copies from the affected node latched indefinitely (harper-pro#699 territory) —
a queue bug surfacing as a storage symptom.

## Why the nominated recovery does not cover it

`queue.claimFloor.resetInterval`'s docstring names the out-of-plugin write paths it exists to
cover — the operations API and the exported REST surface. Residency-routed **in-plugin** writes
are not mentioned, and they are the high-volume path. The reset is also worker-0-gated inside
`syncQueueState` (see the open question in #110).

## Possible directions

1. **Lower the owner's floor, not the writer's.** The write already routes to the owner; the floor
lowering needs to ride with it rather than being applied locally. That is the fix that makes
the invariant unnecessary.
2. **Bound the forward advance** so a floor can never outrun rows the pass never scanned (#110).
That alone would stop the stranding even if the lowering stays local.
3. **Make the below-floor condition observable.** The backlog snapshot already computes
`below_floor`; a node with essentially its whole due backlog below the floor should alarm.
4. Failing 1, at minimum assert the invariant at the funnel — reject or warn on a `writeSchedule`
whose `nextRenderTime` minute is already older than `guard`, so a slow batch fails loudly
instead of silently stranding rows on a peer.

Start at writeSchedule and trace lowerFloorFor through leaseTable().lowerFloorTo(...) and the local coordination.SharedBuffer. Then read resources/Target.js, Sitemap.js, and getResidencyByUrl to compare existing per-URL handling with residency ownership. Done should include an agreed direction, coverage for ordinary in-plugin writes, and evidence that rows are not left below the owning node's floor.

Vom Indexierungsmodell aus dem Issue-Text verfasst.