Hmbown / Hmbown/Codewhale

Sub-agents die mid-task with no warning, no hand-back and uncommitted work: four failure modes observed in one session

Open
#6,194 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
41k
Forks
3.6k
Avg merge
13h 59m
Merged PRs (30d)
299

Description

## Summary

Across one working session, five sub-agent dispatches failed in four distinct
ways. Every failure shared a shape: the parent could not tell what the child had
done, and the workspace was left in a state the parent had to hand-repair. None
of the failures were the child's fault — each one was the runtime withholding
information the child needed or discarding work the child had done.

This is written from the parent's side, with the receipts.

## Mode 1 — a shared token cap starves every parallel child

Four read-only children dispatched under one run budget of 120,000 tokens:

```
explore budget_exhausted 26013 in / 1534 out
planner budget_exhausted 30716 in / 1390 out
explore budget_exhausted 30983 in / 1860 out
planner budget_exhausted 37898 in / 588 out
→ "shared token budget exhausted (124254/120000 measured input + output tokens
across the run)"
```

Each child spent ~30k on **input alone** — reading files — and never reached a
report. A cap sized for one child was a quarter of that for four. Filed as #6189
with the config default at `crates/config/src/lib.rs:2523`.

## Mode 2 — a wall-clock kill mid-edit, with no warning and no hand-back

An `implement` child on the GPUI repository:

```
status: budget_exhausted
failure_class: wall_time_budget
steps: 143
elapsed_ms: 1800002
usage: 10,194,371 input + 41,619 output
"child wall-time budget exhausted during a model request; reporting remains
bounded by the original deadline ... No model hand-back report: the bounded
report deadline expired."
verification: status "claim_mismatch" — observed changes without a change
declaration: src/workspace/history.rs, src/workspace/mod.rs,
src/workspace/render.rs
```

`DEFAULT_CHILD_WALL_TIME = Duration::from_secs(30 * 60)` at
`crates/tui/src/tools/subagent/mod.rs:271` — the kill landed at exactly 30:00.

Three separate defects in one event:

1. **The child was never told it had a deadline.** Nothing in its context said
"30 minutes". It had no way to pace itself, so it began a multi-file refactor
at step ~140 with no time left to finish or commit it.
2. **It was killed during a model request**, so there was no boundary at which to
commit or report. The work was left uncommitted and not compiling — 8 errors
on `cargo check --all-targets`.
3. **The reserved hand-back did not happen.** The message says reporting "remains
bounded by the original deadline" and then that "the bounded report deadline
expired". The reserve exists and bought nothing, the same finding as #6189.

The burn rate is worth noting too: 10.19M input over 143 steps is ~71k input per
step, so the child was carrying a very large context by the end. Long-running
children get quadratically more expensive because each step re-sends everything.

## Mode 3 — declared scope does not match what changed

The same child changed three files it never declared. The runtime *detected*
this (`claim_mismatch`) but the parent had to reconcile it by hand. `write_roots`
is a permission boundary, not a change declaration; nothing makes a child say
what it intends to touch, and nothing reconciles the difference while it still
matters.

## Mode 4 — silent config-shape failures

Three spawn attempts were rejected before starting, each revealing a different
unusable corner:

```
"Fleet member selector `role:general` is ambiguous; choose one member explicitly"
"Custom sub-agent requires a non-empty allowed_tools list"
"write scopes require explicit workspace_write or worktree_write authority"
```

The requested model was `deepseek-flash`, which is only reachable on read-only
roles; the write-capable route could not carry it. That is not discoverable from
the tool schema, so the parent discovers it by failing. These are cheap to fix
and were pure friction.

## What would actually help

Ordered by how much of the observed pain each removes:

1. **Put the deadline in the child's context.** Cheap, and it converts a hard
kill into a paced finish. The child should know its wall clock and its token
allowance from the first turn.
2. **Warn at a fraction of the budget** (say 75%), with an instruction to commit
and report. This is the single change that would have saved the GPUI refactor.
3. **Make the hand-back unlosable.** A child being killed for any budget reason
must still deliver what it has: its last report, the paths it touched, and
whether the tree compiles. "No model hand-back report" must not be a possible
outcome.
4. **Commit discipline as a runtime expectation, not a prompt suggestion.** A
child that has written files but not committed them for a long time is a
signal worth surfacing — to the child, and to the parent.
5. **A declared change set with reconciliation.** Let a child declare intended
paths up front and report actual paths at every step; surface the diff to the
parent while the child is still alive, not only in a post-mortem receipt.
6. **Discoverable selectors and route/model capability.** The spawn schema should
say which roles accept which models and which need which authority, instead of
rejecting after the fact.
7. **Context growth control for long children.** 71k input per step means a long
child costs quadratically. Either bound the scope of long children or compact
their context.

## Reference

`refs/dsh` reads as a working answer to most of this: bounded member limits with
named errors, mailbox outcomes that are always explicit ("accepted" or "retained
as queued", and a queued message must not be resent), a durable record of what
each member did, and `waitForChange` instead of polling. See #6192 for the wider
inventory. The lesson worth taking first is small: **tell the child what it is
racing**.

Contributor guide

Open the contributing guide

Research direction

Start with crates/config/src/lib.rs:2523 and crates/tui/src/tools/subagent/mod.rs:271, then review #6189 and #6192 plus the refs/dsh reference. Reproduce the reported budget, hand-back, change-scope, and spawn-configuration failures before narrowing the work. Done should include an agreed subset of fixes with tests or receipts covering the affected failure mode.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ai-infra-agents, cli, devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.