anthropics / anthropics/claude-code
[BUG] WebFetch truncation is invisible to the model — absent from the tool description, unmarked in the result, and the web-fetch subagent can neither detect nor recover from it
- Dominant language
- Python
- Stars
- 145k
- Forks
- 23.1k
- PR merge metrics
- PR metrics pending
Description
### Preflight Checklist
- [x] I have searched [existing issues](https://github.com/anthropics/claude-code/issues?q=is%3Aissue%20state%3Aopen%20label%3Abug) and this hasn't been reported yet
- [x] This is a single bug report (please file separate reports for different bugs)
- [x] I am using the latest version of Claude Code
### What's Wrong?
`WebFetch` truncates long pages at a fixed character limit. The human documentation now says so, and even says what to do about it — "use `curl` via Bash for the unprocessed page". None of that reaches the model that has to make the decision. The tool description is the model's entire picture of the tool, and it contains no mention of truncation, no limit, and no recovery path.
Here is the description in full, as it appears in the agent's tool definitions — one lead sentence and four bullets, nothing elided:
> Fetches a URL, converts the page to markdown, and answers `prompt` against it using a small fast model.
>
> - Fails on authenticated/private URLs — use an authenticated MCP tool or `gh` for those instead.
> - Fails on localhost and other hostnames without a dot; for a local server, use curl via Bash.
> - HTTP is upgraded to HTTPS. Cross-host redirects are returned to you rather than followed; call again with the redirect URL.
> - Responses are cached for 15 minutes per URL.
The words truncate, limit, size and length do not appear. The input schema is `{url, prompt}` with `additionalProperties: false` — no offset, no page, no max-length. The string "use curl via Bash" *is* present, but it is scoped to unreachable hosts, not to dropped content, and it says nothing about curl returning a more complete page.
So the model has no basis for knowing that a fetch can come back partial, and no reason to reach for the remedy that the docs describe to humans. On a plain-text fetch of `https://www.rfc-editor.org/rfc/rfc9110.txt`, the tool's own boundary note reported returning 39,415 of 502,907 characters — 7.8% of the document. Nothing in the tool result carries that forward as a flag, so the calling model sees an answer about 7.8% of a document presented exactly like an answer about all of it.
The result is that a fetch reporting "the page doesn't mention X" is indistinguishable from the page genuinely not mentioning X. A commenter on #51783 reported precisely this — a 2.5 MB response truncated with no marker, after which the summarizer stated the repository "has no `packages/` directory". That issue was closed for its documentation scope and the observation was never addressed.
### The `web-fetch` subagent makes this structural
There is a built-in `web-fetch` subagent type whose entire toolset is `WebFetch`, plus a handback and an advisor. It has **no Bash**. So when it hits the cut it cannot curl, cannot read a file, and has no fallback of any kind — its only available move is to hand back a caveat.
Two descriptions would need to carry the constraint and neither does:
- **The agent's own prompt.** It is not told that long pages are truncated, so it cannot reliably detect that it happened, and it cannot tell its caller "this page exceeds what I can retrieve, fetch it yourself with curl" — which is the one genuinely useful thing it could say.
- **The caller-facing agent description.** It reads "Use this to fetch and read web pages / URLs when you do not have a direct WebFetch tool of your own", with no mention of a size ceiling. A caller delegating "read this long specification" to it is routing the task to an agent that is structurally incapable of completing it, and cannot know that in advance.
### What this costs in practice
From a background subagent transcript of mine (2026-09-17, v2.1.274), researching the INSDC Feature Table specification:
> **Critical caveat up front:** WebFetch truncates every full copy of the INSDC Feature Table at ~39.4k characters, which lands mid-Appendix-I. Appendix II (feature-key → qualifier lists) and Appendix III (qualifier definitions) both fall past the cut in every complete copy. […] answered only from a **model-generated summary** of the truncated tail. Membership is almost certainly right […] but I cannot present it as the FT's exact text, and I cannot vouch for completeness.
That agent had Bash available and called it 59 times in that same transcript, so this is a measured count rather than an assumption. One `curl` would have returned the entire specification. It inferred the truncation from the shape of what came back, degraded its own answer, warned that it could not vouch for completeness — and never reached for the remedy, because nothing in its tool definition suggested one existed.
### What Should Happen?
Nothing here requires raising the limit.
1. **State the limit and the remedy in the WebFetch tool description** — that long pages are truncated at a fixed character limit, and that `curl` via Bash returns the unprocessed page. This is the same advice the public docs already give humans; the model that needs it is the one that never sees it.
2. **Mark the truncation in the result** — a `truncated: true` flag with characters retained and total, plus a marker in the content handed to the extraction model, so "absent from the page" and "absent from the part I was given" stop being the same observation.
3. **Tell the `web-fetch` subagent in its own prompt** that its content can be truncated and that it has no way to recover, so it can hand back "this exceeds what I can retrieve, fetch it directly" rather than a hedged partial answer.
4. **Say it in the caller-facing `web-fetch` agent description** too, so a caller can choose curl instead of delegating a long-document task to an agent that cannot finish it.
### Error Messages/Logs
```shell
```
### Steps to Reproduce
1. Inspect the WebFetch tool description available to the model. Observe that it states no size limit and gives no recovery path for dropped content.
2. Ask Claude to `WebFetch` `https://www.rfc-editor.org/rfc/rfc9110.txt` and quote the final lines verbatim — the RFC ends with the editors' postal addresses.
3. The answer comes from the first ~39.4k characters; the reported end of the document is wrong, and nothing in the result indicates that 92% was missing.
4. `curl -sS https://www.rfc-editor.org/rfc/rfc9110.txt | wc -m` returns 502,907 characters (502,941 bytes) in one call, with the real tail present.
5. Repeat step 2 by delegating to the `web-fetch` subagent. It has no Bash, so even having inferred the truncation it cannot perform step 4.
### Claude Model
Opus
### Is this a regression?
No, this never worked
### Last Working Version
_No response_
### Claude Code Version
2.1.274
### Platform
Anthropic API
### Operating System
Ubuntu/Debian Linux
### Terminal/Shell
iTerm2
### Additional Information
### One inconsistency worth fixing alongside
The tool description says WebFetch "answers `prompt` against it using a small fast model" — a summarizer. In at least some configurations the system prompt instead states that WebFetch "returns the raw page as markdown inside `` tags rather than a summary". Those describe different return values, and the schema still requires a `prompt` whose stated purpose only makes sense under the summarizer behaviour. A model cannot reason about the fidelity of what it received while its own two sources disagree about what it received.
### Prior art
- **#51783** — the only one closed by a human (bcherny, 2026-08-17), as completed, once the truncation was documented for readers. Its comment thread contains the silent-truncation report described above, which was not addressed. Closed but **unlocked**.
- **#22937** — "WebFetch tool truncates large documentation pages, preventing access to critical content". Asked for a content-length indicator, an error on overflow, and offset/pagination. Labelled **high-priority** by triage, then closed by `github-actions[bot]` after a stale label and locked. No maintainer ever commented.
- **#58467** — optional raw/verbatim mode. Same story: stale-bot closed, bot-locked, no maintainer comment.
- **#50647**, **#59882**, **#53297** — lossy-extraction and fidelity reports. All three stale-bot closed, bot-locked, no maintainer comment.
- **#90416** (open) — structurally identical no-signal failure for PDFs: unreadable content reaches the summarizer with no marker, producing fabricated verbatim quotes.
- **#73514** (open) — the maintainer reply here is the only public source for the raw-passthrough cap (~100 KB / ~25K tokens for preapproved Markdown documentation domains).
To be explicit about the pattern: of the six closed issues in this area, five were closed by the stale bot with no maintainer having commented at any point, and subsequently locked so they cannot be revived. I am filing rather than commenting because #51783 is the only unlocked venue and it is closed as completed.
Already shipped, not being re-requested: the `Accept` header preferring Markdown (#60856, #41097), and the download-deadline and cache-TTL environment variables.
### Environment Info
- Platform: linux (arm64 devcontainer, Ubuntu 26.04, on an Apple Silicon macOS host)
- Terminal: VS Code devcontainer
- Version: 2.1.274
- Feedback ID: 2c12de47-9904-4de3-80f4-e3eac31a337e
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the WebFetch tool definition, result construction, and the built-in web-fetch subagent and caller-facing descriptions. Reproduce the RFC 9110 fetch and compare it with curl output, then trace how truncation reaches the model. Done means the limit and recovery path are described, truncated results are marked, and both subagent descriptions explain the constraint and fallback.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bash, python
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100