anthropics / anthropics/claude-code
[Bug] 529 "never recovers": default retry envelope is ~2.5 min with hard give-up caps and no post-recovery resume; built-in ride-out mode (CLAUDE_CODE_RETRY_WATCHDOG) is undocumented
- Dominant language
- Python
- Stars
- 145k
- Forks
- 23.1k
- PR merge metrics
- PR metrics pending
Description
### Preflight Checklist
- [x] I searched existing issues. Symptom-level 529 reports exist (#81327, #70594, #69767, #78535, #69975, #68502), but none document the actual client retry policy or the existing-but-undocumented mitigation described here. 0 hits for `CLAUDE_CODE_RETRY_WATCHDOG` across issues, repo, and docs.
- [x] Single report: the shipped 529 retry policy makes any overload event longer than ~2.5 minutes indistinguishable from "529 forever", and the built-in ride-out mode is undocumented.
### What's Wrong?
Users keep filing "529 Overloaded never recovers" reports. I reverse-read the retry logic from the shipped **2.1.220** bundle (strings of the distributed binary; identifiers are minified, names below paraphrased) to understand why it *feels* permanent. Findings:
1. **Default per-turn retry envelope is only ~2.5 minutes.** `maxRetries = 10`, backoff `500ms * 2^(attempt-1)` capped at **32s** (+0–25% jitter). Total ≈ 160s. Real overload events routinely last tens of minutes, so every turn attempted during an event exhausts its retries and dies. From the user's chair: every message fails with 529 → "it never recovers".
2. **`Retry-After` > 60s aborts instead of waiting.** If the computed delay exceeds 60_000ms (only reachable via a server `retry-after` header), the client gives up immediately (`tengu_api_retry_after_too_long`) rather than honoring the server's requested wait.
3. **API-key (non-subscription-OAuth) sessions on opus/fable/mythos-class models give up after just 3 × 529** with `Repeated 529 Overloaded errors` — while the status line advertises `attempt N/10`. (With a `fallbackModel` configured, the 3rd 529 instead force-switches the model.) The displayed attempt budget and the real one disagree.
4. **Some internal/background query sources are dropped on the first 529 with no retry at all** (`tengu_api_529_background_dropped`). Related symptom report: #68502.
5. **No self-resume after the API recovers.** Once the turn fails, an unattended session (long autonomous runs, headless pipelines) sits parked on the 529 error indefinitely even though the very next request would succeed. Related: #69975.
6. **A ride-out mode already exists but is undocumented: `CLAUDE_CODE_RETRY_WATCHDOG=1`.** It raises maxRetries to **300**, caps each backoff delay at 5 minutes, and bypasses (2), (3) and (4) — i.e., exactly the "keep retrying until the event ends" behavior that the dozens of 529 issues are asking for. It appears nowhere in the docs, README, or changelog. `CLAUDE_CODE_MAX_RETRIES` (clamped to 15 unless watchdog mode) is likewise undocumented.
### Expected Behavior
Either (a) document `CLAUDE_CODE_RETRY_WATCHDOG` / `CLAUDE_CODE_MAX_RETRIES` so users hitting multi-minute overload events can opt into riding them out, or (b) make the interactive default closer to watchdog behavior (the `Retrying in Xs (resets at …) · attempt N/M` banner already communicates long waits well), and ideally (c) resume the interrupted turn automatically once a request succeeds again instead of requiring a human to type `continue`.
### Environment
- Claude Code 2.1.220, native binary, Windows 11 (10.0.26200)
- Auth: subscription OAuth
- Analysis basis: string-level read of the shipped bundle; happy to share exact offsets/snippets if useful.
Contributor guide
No contributing guide indexed for this repository
Research direction
The report is based on the shipped Claude Code 2.1.220 bundle and names the retry behavior plus CLAUDE_CODE_RETRY_WATCHDOG and CLAUDE_CODE_MAX_RETRIES. Start by locating the retry implementation or documentation entry points corresponding to those names, then verify the 529, Retry-After, fallback, background-query, and post-recovery cases. Done means either documenting the existing controls or implementing an agreed retry/resume behavior with coverage for the reported cases.
Written by the indexing model from the issue text.
Assessment
- Domain
- api, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100