Expose structural completion evidence (todo ledger, final-message presence, stop facts) for subagent results

Expose structural completion evidence (todo ledger, final-message presence, stop facts) for subagent results

Summary: Subagent turns that end incompletely (provider length cuts, model premature stops) are reported as status: completed whenever a final message exists. The runtime already tracks structural completion evidence (todo ledger, stop facts, final-message presence) that would let parents distinguish a real handoff from a cut — without resurrecting any content-length heuristic.

1. Problem

A subagent result is judged complete by the parent tool result when the subagent produced a final message — but nothing in the result surface reflects how the turn ended. A turn that was cut can still be reported status: completed, so parents and users cannot tell a genuine handoff from a cut one.

The practical consequence is that parent logic which acts on status: completed inherits the gap: it proceeds on state the subagent never actually produced, and the failure surfaces later as a downstream symptom with no link back to the cut turn. The 62-char case below is exactly that — a summary that reads like a handoff but announces work that was never run.

Observed concrete cases (paraphrased):

• A 62-character degenerate summary, roughly "The gate passes with the marker detected. Now the S6 tests:", was accepted as a completed summary. The run it summarized never happened; the turn had already been terminated.
• A 188-character mid-thought fragment, roughly "Now let me confirm how per-wave reportJson gets nodeReports...", served as the entire final message of a prematurely stopped turn.
• A whitespace-only final message — nothing but a blank line where a handoff was due — also passed the gate as completed.

These passed the existing completion gate because a final message merely existed. The parent then acted on state the subagent never produced.

2. Observed wire evidence

Offline evaluation over 2,231 real subagent turns from wire logs (corpus snapshot 2026-09-07). Turns were labeled only when the evidence was unambiguous; everything else was marked ambiguous and excluded from rates.

Set	Count
Subagent turns evaluated	2,231
Clearly incomplete	149
Clearly complete	45
Ambiguous (excluded from rates)	2,037

Breakdown of the 149 clearly-incomplete turns:

Kind	Count
max_tokens cuts at exactly 32,768 output tokens (a Ling-family coder model via OpenRouter)	146
Premature stops by explore agents (a DeepSeek-family explore model via a direct provider) — a 188-char mid-thought final, a whitespace-only final, and a 62-char degenerate summary that passed as completed	3

All 146 cuts land exactly at the 32,768-token output cap — the provider stopped generation, not the model — which is unambiguous evidence the turn was still producing when it ended. "Clearly complete" was labeled only where the turn finished with its own natural stop and a substantive handoff; "ambiguous" covers everything else, including all short-but-plausibly-finished finals.

3. Why content-length heuristics are insufficient

The removed <200-char policy (reimplemented offline as a baseline over the same corpus) catches only 43.0% of genuinely cut turns (64/149) and misses the other 57% (85/149) — typically long degenerate generations that pad well past 200 characters before the provider length cut lands. Its 0.0% false-intervention rate over the 45 clearly-complete turns is real, but the heuristic trades away most of the recall for it.

PR #3459 removed that policy for good reasons: it bounces legitimately short answers (a correct "No" or a two-line handoff is not an incomplete turn), it punishes style rather than substance, and its threshold is arbitrary. Restoring any length threshold would reintroduce those false positives while still missing most genuine cuts. The information needed to judge completion does not live in the final message's length — it lives in whether the turn's own bookkeeping says it finished. In short: length is a property of the final message; completion is a property of the turn.

4. Structural evidence already exists

PR #3459 already moved the result surface in this direction — structured stop_reason and resume_hint, and removal of the length bounce. This issue is the natural next step: turn what the runtime already records durably into evidence parents can branch on.

The runtime already tracks, durably, the facts needed to distinguish complete from cut:

• TodoList state is persisted under the tools.update_store records (todo key) — an open-todo ledger at termination is structural evidence of unfinished work.
• The parent-facing Agent result already carries stop_reason + resume_hint (PR #3459), but only as unstructured hints for display/resume — not as facts the parent can branch on.
• Final-message presence and per-turn tool traces exist per agent turn.
• Subagent final messages and per-agent wire files already exist under the session directory — but they are documented debug material. The ask here is to expose the evidence through a supported surface, not to have users parse wire files.

Candidate rule set — structural facts only, no length, no semantics:

1. stop-abnormal (stop reason indicates abnormal termination: provider length cut, error, etc.) → INCOMPLETE
2. todo-open (ledger has open items at termination) → INCOMPLETE
3. final-absent (no final message) → INCOMPLETE
4. todo-clean (ledger present and empty at termination) → COMPLETE
5. Otherwise (no ledger) → UNVERIFIABLE

Presented as a decision procedure: evaluate in order, first hit decides. The corpus counts below are per-rule occurrences, not final verdicts — a turn can trip several rules at once.

Results over the same corpus (baseline vs structural):

Metric	Baseline (<200-char)	Structural candidate
Recall on incomplete	43.0% (64/149)	99.3% (148/149)
False intervention rate on complete	0.0%	0.0% (0/45)
Interventions	64	148

Rule hit counts over the full 2,231-turn corpus:

Rule hit	Outcome	Count
stop-abnormal	INCOMPLETE	146
todo-open (across corpus)	INCOMPLETE	163
final-absent	INCOMPLETE	66
todo-clean	COMPLETE	264
no-ledger	UNVERIFIABLE	1,587

The single structural miss is the no-ledger explore agent's 188-char premature stop — UNVERIFIABLE by design, because with no ledger there is nothing structural to judge. That is exactly the case the length baseline happens to catch. This limitation is inherent to evidence-free turns; see the proposal in §6.

Estimated cost per automatic resume (offline approximation): ~21.8k output tokens (structural) vs ~17.0k (baseline). The higher figure is not overhead — structural catching more genuinely cut turns means more resumes, each resuming real work that would otherwise have been acted on as complete.

Corpus-composition caveat: the labeled sessions over-represent cut-era sessions of the Ling-family coder model, so the 76.3% intervention rate implied here is a corpus property, not a production expectation.

5. Limitations (be honest)

• No-ledger blind spot. Turns from agents/profiles that never maintain a todo ledger carry no structural completion evidence. They are UNVERIFIABLE under this rule set, and UNVERIFIABLE is the correct first-class outcome for them — it means "route to a parent decision," not "assume complete." A length heuristic happens to catch some of these, at the cost of bouncing short legit answers.
• UNVERIFIABLE must be a first-class label. The value of this design depends on parents treating UNVERIFIABLE as distinct from both COMPLETE and INCOMPLETE rather than collapsing it.
• Corpus composition. Labeled sessions skew toward cut-era Ling-family sessions; rates in §4 do not generalize to production traffic without re-measurement.
• Cost estimate is an offline approximation. Resume-cost-per-token figures come from logged token counts, not from live resume runs.
• No content judgement. Structural rules say whether bookkeeping finished, not whether the answer is good. Parents still own quality checks.
• FIR measured only on labeled turns. The 0.0% false-intervention rate covers only the 45 clearly-complete turns; neither candidate was FIR-checked against the 2,037 ambiguous turns.

6. Minimal API proposal

No full "completion oracle" is needed. The evidence exists; the ask is to expose it through a supported surface. Out of scope for this request: any runtime-side automatic action (auto-resume, auto-veto). The proposal only surfaces evidence; the decision remains with the parent or an external adapter.

1. Expose structural completion evidence on the Agent tool result (or via hook payloads): final-message presence, open-todo count at termination, and per-turn stop facts. Concretely, that is three small fields — a final-message-present boolean, an open-todo count, and the stop reason the runtime already records — no new machinery. Parents can then branch on COMPLETE / INCOMPLETE / UNVERIFIABLE themselves; the labeling rule set above is a reference, not a requirement.
2. Include unfulfilled-task context in resume_hint. The resume path already exists from PR #3459; scope it to carry what was left open (e.g., open todo items at termination) so a resume starts from the ledger, not from a generic "continue" prompt. The todo items are already persisted; the hint only needs to reference them.
3. Optional progress-ledger capability for long-running/multi-step subagents — a supported way for any subagent to record structured progress, not hardwired to specific built-in profiles. Agents that opt in become verifiable; agents that do not remain UNVERIFIABLE. This is the only item that asks for a new capability rather than exposure of existing state — it is strictly optional and independent of items 1–2.
4. Include the agent instance identifier in hook payloads. SubagentStart / SubagentStop / Stop currently expose only the profile name, so parallel same-profile subagents cannot be distinguished by external adapters — which also blocks correlating hook events with per-turn structural facts.

7. Related

• PR #3459 — foundation this builds on: it removed the length-bounce policy and added structured stop_reason + resume_hint to subagent results.
• Claude Code's TaskCompleted hook precedent — a runtime veto over a completion claim, with stderr fed back into the result; the same shape of "the consumer decides whether the claim holds" applies here.
• The proposer runs a measured, observe-only local adapter (hooks-only, feature-flagged, no behavior change) and is happy to share the data and tooling with maintainers.

Reproducibility

The evaluation tooling is a small stdlib-only Python script run offline over captured wire logs; the author will share it on request, along with the anonymized corpus snapshot and labeling methodology.

Start by reading PR #3459 and the existing Agent tool result, SubagentStart, SubagentStop, and Stop hook payloads. Trace where stop_reason, resume_hint, todo records, and final-message state are recorded. Done means exposing the proposed structural fields and identifiers through a supported surface without adding automatic resume or content-length heuristics.