feat(context): enforce wave-boundary checkpoint+reset and lossless intra-wave compaction (PreCompact flush + WASM gates)

## Summary

Long autonomous runs repeatedly exhaust the context window and rely on **reactive, harness-driven auto-compaction mid-wave**, which silently loses pipeline continuity and forces expensive re-derivation of state. The factory externalizes state to `STATE.md` + the `factory-artifacts` branch, but it does **not** use that externalized state as a deliberate context-management mechanism: there is no enforced wave-boundary checkpoint+reset, and **no `PreCompact` hook at all**, so when the harness compacts mid-wave the plugin is not in the loop.

This proposes a two-part, enforced strategy:

- **(A) Cross-wave: hard reset.** At each wave close, write a *verified* handoff to external state, then reset the session and re-hydrate only the next wave's slice.
- **(B) Intra-wave: make auto-compaction lossless.** A `PreCompact` flush + retention-pinning, tool-result clearing, sub-agent isolation, and a proactive threshold — so mid-wave compaction never drops load-bearing facts.

Both parts are **enforced via hooks**, split along the factory's existing convention (WASM for deterministic gates, shell for effectful flush).

## Evidence (from a week of orchestrator sessions)

- **wirerust** flagship run auto-compacted **14×** ("This session is being continued… ran out of context" at multiple points); STATE.md repeatedly bumped its limits (`515→199 lines` compaction; `202 (2 over soft target)`). Each event forced re-reading specs and re-deriving wave position. The user repeatedly issued "make sure state is up to date so we can clear the session and start fresh."
- **engineering-report** ran adversarial loops to **30 and 23 passes**, with context overflow occurring in exactly those long-loop sessions.
- **jira-cli** showed the downstream risk of trusting in-context memory over external truth: fabricated SHAs / a non-existent "56/1 flake" story typed into state-manager prompts ("again pre-typed wrong values… fabricated SHAs caused a stale-base PR"). A lossy in-session summarizer would amplify exactly this.

## Terminology collision to fix

The repo currently overloads "compaction":
- **Context compaction** — the harness summarizing/clearing the LLM context window (what runs out and auto-fires).
- **State compaction** — the `compact-state` skill slimming `STATE.md` to <200 lines (a *file* operation).

These are unrelated today. The `PreCompact` hook is what should *tie them together*: when a context compaction is imminent, trigger a state flush first.

## Current state (`1.0.0-rc.20`)

- Hook events registered: 45 PostToolUse, 19 PreToolUse, 2 SessionStart, 2 SessionEnd, 3 Stop, 7 SubagentStop — **0 PreCompact**. The plugin has no integration with the harness compaction event.
- `compact-state` slims the STATE.md file; it does nothing for the context window.
- `state-manager` + `state-burst` + `next-step` + `recover-state` + `wave-state.yaml` provide the bones for checkpoint/resume, but nothing enforces a checkpoint **at the wave boundary** or a scoped re-hydration after reset.

## Research basis

External research (Perplexity Sonar deep-research, plus the sources below) lands decisively on a hybrid with reset as the primary cross-wave mechanism:

> "the balance of evidence favors using **hard session resets at wave boundaries** as your primary context-management strategy, supplemented by carefully engineered external state and judicious intra-wave compaction."

> "no wave boundary is crossed without a hard reset and external checkpoint."

Why continuous in-session compaction is the *weaker* cross-wave mechanism (and dangerous for a *verified* pipeline):
- **Stacked lossy summaries drift:** "By resetting instead of repeatedly compacting, you avoid stacking multiple lossy summarization passes on top of one another."
- **Hallucinated state:** "a summarizer may misinterpret a partial success as a full success, stating that 'all tests passed' … there may be no in-session way to check its accuracy." (already a live failure mode here — see jira-cli evidence)
- **Determinism/observability:** resets let you "rerun the agent from that checkpoint with a fresh session, without relying on any opaque provider compaction logic."
- **The precondition is already met:** reset is the better choice precisely *when state is externalized* — which the factory already does.

Compaction's correct role is **within** a wave, with the rule: **clear** re-fetchable bulky tool outputs; **summarize** only non-re-fetchable dialogue/reasoning; **anchor** system prompt, CLAUDE.md, architecture, and the STATE.md pointer (exploit prompt caching on that stable prefix).

## Proposal

### Part A — Cross-wave: verified checkpoint → reset → scoped rehydrate

1. **Wave-close checkpoint (verified).** Before a wave is declared done, flush everything the next wave needs into a structured handoff on `factory-artifacts`: decisions (each citing a commit hash / test ID / file path), pending fixes, open process-gaps, last *verified* develop SHA, active BC contracts, next-wave story list. Claims are **verified against git/tests, not memory** (directly mitigates the fabrication failure mode).
2. **Reset.** Recommend/trigger a session clear at the boundary rather than letting auto-compaction carry across it.
3. **Scoped rehydrate.** On the next wave, load **only** that wave's slice (its stories + the specs they touch), on-demand/RAG-style — not the whole spec corpus. Keep `STATE.md` compact; rely on prompt caching for the stable prefix.

### Part B — Intra-wave: make auto-compaction lossless and rare

1. **`PreCompact` flush (NEW).** On imminent context compaction, run a `state-burst` to persist wave-critical state first, and supply **retention instructions** for what the summarizer must preserve (wave id, active BC contracts, last verified SHA, STATE.md pointer, anchored CLAUDE.md/architecture).
2. **Tool-result clearing.** Prefer clearing re-fetchable bulky outputs (test logs, large file dumps) over summarizing them.
3. **Sub-agent isolation.** Route heavy operations (large test runs, broad greps, research) to sub-agents so their cost never enters the orchestrator window. (Already a factory strength — enforce it.)
4. **Proactive threshold.** Checkpoint at ~50–70% window usage rather than waiting for last-second (~95%) auto-compaction.

### Part C — Enforcement (hook split, matching the factory's existing convention)

The factory already splits hooks cleanly: **WASM** for deterministic, parse-heavy *validators* (`validate-state-structure`, `validate-burst-log`, `validate-stable-anchors`, `regression-gate`, `pr-manager-completion-guard`, `handoff-validator`) and **shell** for *effectful* guards (`check-factory-commit.sh`, `factory-branch-guard.sh`, `red-gate.sh`). Apply the same rule:

| Hook | Binary/shell | Rationale / reuse |
|---|---|---|
| `PreCompact` → flush `state-burst` + commit | **Shell** | Effectful (git/FS); WASM is sandboxed. Joins `check-factory-commit.sh` family. |
| Checkpoint-completeness gate ("handoff has all required fields before reset") | **WASM** | Correctness-critical, parse-heavy. Extend **`handoff-validator.wasm`** / `validate-stable-anchors.wasm`. |
| Wave-boundary reset-blocker ("no reset until checkpoint CLEAN") | **WASM** | Same class as `regression-gate.wasm` / `pr-manager-completion-guard.wasm`. |
| PreToolUse delegation guard ("heavy op → must delegate to sub-agent") | **WASM** | Deterministic pattern decision; tamper-resistant; identical macOS/CI. |
| PostToolUse output-size guard (nudge tool-result clearing) | **WASM** (shell acceptable) | Lean WASM for portability. |
| Proactive-threshold config | settings, not a hook | e.g. configure auto-compact band. |

**Why WASM for the gates specifically:** the sessions exhibited a recurring portability-bug class (macOS BSD vs CI GNU `date`/`sed`, absolute-path handling). An integrity gate that silently mis-fires on coreutils differences is worse than no gate — exactly why `validate-state-structure` is already WASM.

**Caveats (don't over-WASM):** WASM can't do git/FS, so effectful work stays shell (WASM decides → shell acts). Binary hooks raise contribution friction and can't be hot-patched, so reserve them for correctness-critical gates and trivially keep nudges in shell. New WASM blobs must ride the existing reproducible build + checksum pipeline (the dual `foo.wasm` / `foo_underscore.wasm` artifacts suggest one already exists) so they remain auditable.

## Acceptance criteria (draft)

- [ ] A wave cannot be closed until a **verified** handoff (required fields present, claims cross-checked against git/tests) is written to `factory-artifacts` — enforced by a WASM checkpoint-completeness gate.
- [ ] The pipeline performs an explicit **session reset at wave boundaries** and re-hydrates only the next wave's scoped slice (not the full spec corpus).
- [ ] A **`PreCompact` hook** flushes wave-critical state and pins retention instructions before any context compaction; mid-wave compaction no longer loses load-bearing decisions/SHAs.
- [ ] Bulky, re-fetchable tool outputs are cleared rather than summarized; heavy ops are delegated to sub-agents (guarded).
- [ ] "Context compaction" and "state compaction" are disambiguated in docs; `compact-state` is clearly the *file* operation and the `PreCompact` hook is the *window* integration.
- [ ] Single-wave runs and short pipelines incur no new friction (reset/checkpoint scales with wave boundaries, not every step).

## Open questions

- Proactive threshold value (50% vs 70%) and whether it's configurable per-autonomy-level.
- Should the wave-boundary reset be **automatic** (orchestrator self-clears) or **prompt-the-human** ("wave N closed and checkpointed — clear and start wave N+1?")?
- Can a `PreCompact` hook in the current harness *block/defer* compaction until the flush completes, or only run alongside it? (Affects whether the flush is a hard guarantee or best-effort.)
- Scoped rehydration: retrieval/RAG over the spec corpus vs. a curated per-wave manifest in `wave-state.yaml`.

## Notes / relationships

- Composes with #171 — deferred process-gaps (PG-W9/W12/W17 class) belong in the **handoff/state**, not carried in the orchestrator's head across resets.
- Reduces the blast radius of the fabrication failure mode seen in jira-cli by making external, verified state the source of truth at every boundary.

## Sources

- Anthropic — Effective context engineering for AI agents: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
- Anthropic — Context windows: https://platform.claude.com/docs/en/build-with-claude/context-windows
- Anthropic — Tool-use / context-engineering cookbook: https://platform.claude.com/cookbook/tool-use-context-engineering
- Anthropic — Prompt caching: https://platform.claude.com/docs/en/build-with-claude/prompt-caching
- Claude Code — Context window docs: https://code.claude.com/docs/en/context-window
- LangGraph — Persistence / checkpointing: https://docs.langchain.com/oss/python/langgraph/persistence
- LangChain — Memory concepts: https://docs.langchain.com/oss/python/concepts/memory
- Microsoft Agent Framework — Compaction: https://learn.microsoft.com/en-us/agent-framework/agents/conversations/compaction
- "AI agent failure modes beyond hallucination" (compaction loss, context rot): https://dev.to/maximsaplin/ai-agent-failure-modes-beyond-hallucination-208g
- Context rot (lost-in-the-middle / recency): https://www.producttalk.org/context-rot/
- XTrace — AI agent context handoff (structured briefing vs dump): https://xtrace.ai/blog/ai-agent-context-handoff
- OpenClaw — context-loss techniques (pre-compaction memory flush): https://codepointer.substack.com/p/openclaw-stop-losing-context-8-techniques
- Stack Overflow — Reliability for unreliable LLMs (durable execution / idempotent state): https://stackoverflow.blog/2025/06/30/reliability-for-unreliable-llms/

---

*Filed after mining a week of orchestrator sessions (jira-cli / wirerust / engineering-report) for context-exhaustion friction, and validating the strategy with external research. Happy to prototype the `PreCompact` shell flush + the WASM checkpoint-completeness gate first.*