github / github/copilot-cli

Parallel explore subagent fan-out dies to per-model 429s: explore's default model is the only rate-limited one, no backoff, no auto model switch despite eligibleForAutoSwitch

Open
#4,416 0 comments 0 reactions 0 assignees View on GitHub
area:agents area:models
Dominant language
Shell
Stars
11.2k
Forks
1.9k
Avg merge
14h 16m
Merged PRs (30d)
6

Description

### Describe the bug

Launching many subagents in parallel via the task tool concentrates all their model calls on one model bucket — `explore` agents all default to the same lightweight model (currently claude-haiku-4.5). That model appears to have a much tighter per-model burst limit than any other model, so a 16-agent explore fan-out hits HTTP 429 within ~20 seconds. Every subagent then fails with repeated `integration_rate_limited` errors and completes with empty output, while the parent session (on a different model) continues unaffected.

Three compounding problems:

1. **The built-in `explore` agent defaults to the only model that rate-limits under fan-out.** A survey of 1,089 local CLI sessions (~2.7 GB of events.jsonl) found exactly 5 sessions that ever logged `errorType: rate_limit` — every significant incident was a 16x claude-haiku-4.5 explore fan-out. Other models never triggered it despite far heavier use: gpt-5.6-sol (19k assistant messages, 201 sessions), claude-fable-5 (14k messages, including a clean 16-way fan-out re-run of the exact workload that failed on haiku), claude-sonnet-5 (9k), claude-opus-5 (7k), gpt-5.6-terra (5k) — zero incidents.
2. **No backoff.** The agentic loop retried into the same window at ~2 failing requests/second: 182 `integration_rate_limited` errors logged in 87 seconds in one incident.
3. **No model fallback.** The 429 response carries `"eligibleForAutoSwitch": true`, but subagents never switch — they just die and return empty results, silently wasting the whole fan-out.

Sample error from events.jsonl:

```json
{"errorType":"rate_limit",
"message":"You've hit the rate limit for this model. Please switch models or wait for your limit to reset in under a minute. Learn More (https://docs.github.com/copilot/concepts/rate-limits). (Request ID: C33A:2A98BA:2871A5:13FF8A5:6A7702DD)",
"statusCode":429,
"errorCode":"integration_rate_limited",
"eligibleForAutoSwitch":true}
```

Observed concurrency data for the explore default model (same account, 4 incidents across 3 days):
- 16 parallel explore agents -> first 429 in ~20 s, all 16 stall and return empty
- 8 parallel -> clean
- ~15 launched sequentially over 10 minutes with overlap -> 2 transient errors, work completed
- 16 parallel on claude-fable-5 (model override) -> zero errors

### Affected version

1.0.79-5 (Windows x64); incidents also observed on earlier 1.0.7x builds.

### Steps to reproduce the behavior

1. In a session, use the task tool to launch ~16 `explore` subagents simultaneously (all default to the lightweight model).
2. Within ~20 seconds, subagent transcripts fill with "Limit reached — Resets in under a minute"; agents go idle after emitting only setup text and return empty results.
3. Re-run the identical fan-out with a `model` override to a larger model — it completes cleanly.

### Expected behavior

Any (ideally all) of:
- Subagents honor `eligibleForAutoSwitch` and fall back to another model instead of dying (related: #2840).
- The agentic loop backs off per the reset window instead of retrying ~2x/s into the same limited minute (related: #2760).
- The CLI throttles subagent fan-out concurrency per model client-side, since it knows how many concurrent loops it is aiming at one bucket (related: #2545).
- The built-in `explore` agent''s default model either gets burst headroom matching the "fast, lightweight, fan out in parallel" positioning, or the CLI spreads large fan-outs across multiple eligible lightweight models.

### Additional context

- The failure is invisible from the parent''s perspective until results come back empty: `subagent.completed` events fire normally with ~200-byte payloads.
- Nudging stalled subagents (`write_agent`) during the limited minute makes it worse — each nudge adds more 429s to the same window.
- Request IDs from multiple incidents available on request.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure with the task tool using a 16-agent explore fan-out, then compare events.jsonl for the default model and a model override. Done should prevent repeated 429 retries and ensure subagents return useful results through backoff, throttling, or model switching; the payload names no implementation file or test.

Written by the indexing model from the issue text.

Assessment

Domain
ai-infra-agents, cli, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.