world-postgres: startup recovery re-enqueues parked runs and accumulates duplicate jobs

Summary

@workflow/world-postgres treats every pending or running run as runnable during startup recovery.

createWorld().start() calls reenqueueActiveRuns(), which lists every active run and queues it without inspecting its persisted suspension state. A run that is durably parked on an unresolved hook or a future wait is therefore replayed on every process restart.

The recovery enqueue also gets a fresh message ID/job key when no idempotency key is supplied, so repeated restarts accumulate another outstanding Graphile job for the same run.

This reproduces in a single application with one namespace and one Postgres database. It is separate from #2978, which covers cross-application namespace isolation.

Versions

• @workflow/world-postgres@5.0.0-beta.28
• @workflow/world@5.0.0-beta.22
• graphile-worker@0.16.6
• Node.js 24

The same paths are still present on main at 62d570ed4bf38db333ae9fe9ba513c0d6a9d6b91:

• packages/world-postgres/src/index.ts: start() calls reenqueueActiveRuns(storage.runs, queue.queue, ...)
• packages/world/src/recovery.ts: recovery lists all pending and running runs
• packages/world-postgres/src/queue.ts: an enqueue without an explicit idempotency key uses a fresh message ID as its Graphile job key

Reproduction

Using a real Postgres database and the package's official migrations:

1. Persist 500 running runs whose only live state is an unresolved hook. They have no active step, received hook delivery, due wait, or existing queue job.
2. Persist 20 genuinely runnable runs with an active step.
3. Add one deliberately ambiguous active run.
4. Start and close the Postgres World once.
5. Inspect graphile_worker._private_jobs.

Observed: all 521 active runs get a job instead of only the 20 runnable runs.

A second fixture with 20 runnable runs shows the deduplication problem:

1. Start and close the same World five times without consuming the jobs.
2. Inspect outstanding Graphile jobs grouped by runId.

Observed: 100 jobs total, with five jobs per run. Expected: 20 jobs total, with at most one outstanding recovery job per run.

There is also a delivery/restart race: if a durable wake event and its normal delivery job are created around startup recovery, the same run can receive both a normal wake job and a recovery job because they do not share a persisted idempotency identity.

Expected behavior

• A durably suspended run should not be replayed merely because its top-level status is running.
• Persisted runnable work must still recover, including the crash window after a wake event is stored but before its queue job is inserted.
• Repeated startup recovery must not add more than one equivalent outstanding job for the same run/recovery cause.
• If persisted state cannot be classified safely, startup should report/fail rather than replay every active run.

At minimum, recovery needs a persisted runnable predicate plus durable queue deduplication. A stable key derived from the run and the persisted wake/event identity would cover repeated restarts without suppressing later legitimate wakes.

Impact

A service with many long-lived hook-backed sessions can enqueue hundreds of unnecessary replays on every container restart. Repeated restarts multiply the backlog, increase concurrent event-log replay, and can cause long startup delays or memory exhaustion.

Related issues

• #2978 covers unscoped recovery across applications/namespaces; this issue reproduces with only one application.
• #679 and #1531 motivated restart recovery, but do not cover over-selection or duplicate recovery jobs.

Start with packages/world-postgres/src/index.ts, packages/world/src/recovery.ts, and packages/world-postgres/src/queue.ts to trace startup recovery, runnable-state selection, and Graphile job keys. Reproduce the single-namespace restart cases against the official Postgres migrations and inspect graphile_worker._private_jobs. Done means suspended runs are not replayed, persisted runnable work recovers, and repeated recovery does not create duplicate outstanding jobs.