Proposal: sharded map-reduce review for large diffs — k concurrent background tasks + a cross-shard integration pass (working prototype, ~4.3× measured)

Problem

A single /codex:review / adversarial-review is single-threaded end-to-end. On a real 24-file, ~1.8k-changed-line multi-workstream diff, one review ran 40+ minutes and its worker died silently mid-run — total loss, nothing recoverable. Large diffs are exactly where users most need review help, and exactly where one sequential run is slowest and most fragile. #68 asks for parallel reviews directly; #517 / #540 / #509 document the silent-death modes that make a long single run risky.

What we built (working prototype, zero plugin changes)

A Claude Code user skill that orchestrates the existing companion CLI:

1. Gate. Below ~300 changed lines or 8 files → one plain adversarial-review (parallel costs ~k× tokens; small diffs don't earn it).
2. Map. Shard the diff by directory/file-ownership into k ≤ 4 task --background jobs; a directory that dwarfs the rest is split per-file and re-balanced so no shard becomes the bottleneck. Every shard receives the same shared invariant list but attacks only its own files (read-only repo access); each prompt embeds only that shard's hunks plus the review-output schema.
3. Supervision. A watcher reads per-job files (not state.json), verifies the worker pid is actually alive, detects log-mtime stalls, and on death cancels + --resume-last so the thread keeps the analysis it already did. Each shard runs under its own synthetic CODEX_COMPANION_SESSION_ID namespace — that is what makes --resume-last deterministic when several tasks are in flight.
4. Reduce (mandatory). One cheap (--effort low) integration task over the merged findings + mechanical seam hints (imports and style tokens that cross shard boundaries) + a fixed seam checklist. It confirms/rejects every finding and hunts cross-shard defects no single shard could see whole.
5. Output. One merged, deduped, ranked report with per-shard wall-times.

Measured results (real production repo, real historical diff)

The single most valuable finding was cross-shard: one shard changed a global CSS token to a translucent value, silently breaking bg-*/NN opacity-modifier consumers in files owned by other shards (effective alpha 5–7%). A monolithic review can see that class; blind shards cannot — the shared-invariant + reduce design is what recovers it.

Why bring this upstream

Demand exists (#68), and today an orchestrator must engineer around known sharp edges instead of fixing them:

• The shared broker is single-tenant; concurrency works only via the BROKER_BUSY → dedicated-app-server fallback. It works, but it's implicit and undocumented.
• --resume-last is session-scoped and picks the newest resumable thread, so recovering one specific dead task among several requires synthetic session namespaces. A first-class task --resume <thread-id> would remove the hack.
• state.json concurrent writers race and can wipe job state (#286, #517) — we cope by reading per-job files and staggering spawns.
• status trusts the broker's "running" with no pid-liveness check (#517), so host-killed jobs look alive forever; our watcher re-checks the pid.

A native parallel-review subcommand could reuse the existing infrastructure (enqueueBackgroundTask, per-job files, the review schema, the adversarial prompt) and solve these properly instead of routing around them.

Proposed shape

• codex-companion.mjs parallel-review [--base <ref>] [--max-shards 4] [--scope auto|working-tree|branch] plus a /codex:parallel-review command doc; heuristic gate falls back to the existing single review.
• Internals: shard planner (gate + directory sharding + oversize split + seam-hint extraction), per-shard background tasks in isolated session namespaces, watcher with pid/stall recovery, one mandatory reduce turn, merged JSON report.
• Enabling primitives worth landing first (each valuable standalone): task --resume <thread-id>; pid-liveness in status; atomic/locked state writes (also fixes #286).

I'm sending a PR that ports the prototype's core (shard planner, supervision loop, reduce pass) into the companion as a parallel-review subcommand — will link it here. Happy to split it into the standalone primitives above if maintainers prefer smaller slices.

Start at codex-companion.mjs, the existing enqueueBackgroundTask infrastructure, and the adversarial-review entry point. Review the proposed parallel-review command doc and standalone primitives, then verify that sharding, supervision and the mandatory reduce pass produce one merged report while preserving the existing small-diff fallback.