cloudflare / cloudflare/cloudflare-os
webFetch's 30s timeout does not cover the response body read
- Dominant language
- TypeScript
- Stars
- 9.9k
- Forks
- 1.2k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 107
Description
## Summary
`webFetch` declares a 30s timeout, but the timer is cancelled as soon as response *headers* arrive,
so it never bounds the body read. A server that answers immediately and then trickles the body keeps
the agent turn pending indefinitely, and the user's Stop cannot end it either, because the abort
signal the agent framework hands the tool is not passed on.
## Why the timer is dead by the time it matters
[`web-fetch.ts#L271-L295`](https://github.com/cloudflare/cloudflare-os/blob/e1ab8fbd4f609aff7ede9d490bafe1bcf9b2a682/packages/workshop-backend/src/web-fetch.ts#L271-L295)
arms the timer, `await fetch(...)` resolves once headers are in, and the `finally` clears it. The
body is only read afterwards, at
[`#L316`](https://github.com/cloudflare/cloudflare-os/blob/e1ab8fbd4f609aff7ede9d490bafe1bcf9b2a682/packages/workshop-backend/src/web-fetch.ts#L316),
by [`readBodyCapped`](https://github.com/cloudflare/cloudflare-os/blob/e1ab8fbd4f609aff7ede9d490bafe1bcf9b2a682/packages/workshop-backend/src/web-fetch.ts#L85-L137),
which is a bare `while (true) { await reader.read() }` with no signal and no deadline. Nothing
between the two re-arms anything, and `AbortController` cannot fire after `clearTimeout`.
The 1 MiB cap does not help: it bounds total bytes, not time, and a server that stalls below the cap
never reaches it.
## Reproduced on workerd
A server that flushes headers and then writes one byte every 2s, against four variants of the same
code with the timeout shortened to 5s:
| variant | result |
| --- | --- |
| `clearTimeout` in `finally` after `fetch` — **what the code does today** | still pending when the client gave up at **25s** |
| timer cleared only after the body read | `AbortError` after **5057ms** |
| `signal: AbortSignal.timeout(5000)`, no manual timer | `TimeoutError: The operation was aborted due to timeout` after **5046ms** |
| the patch below | `Error: Fetch timed out after 5000ms` after **5039ms** |
So the abort *does* reach an in-flight `reader.read()` in workerd — the mechanism works, it is just
switched off before the body is touched. Note the middle two rows: simply keeping the timer alive
stops the hang but surfaces a raw abort, because the `catch` that produces the friendly message
wraps only the `fetch` call.
worker used for the table
```js
const TARGET = "http://127.0.0.1:18791/"; // flushes headers, then one byte / 2s, never ends
const TIMEOUT_MS = 5000;
async function variantCurrent() {
const ac = new AbortController();
const id = setTimeout(() => ac.abort(), TIMEOUT_MS);
let response;
try { response = await fetch(TARGET, { signal: ac.signal }); }
finally { clearTimeout(id); } // <-- today's behaviour
return await readBodyCapped(response, 1024 * 1024);
}
async function variantKept() {
const ac = new AbortController();
const id = setTimeout(() => ac.abort(), TIMEOUT_MS);
try {
const response = await fetch(TARGET, { signal: ac.signal });
return await readBodyCapped(response, 1024 * 1024);
} finally { clearTimeout(id); }
}
```
## The runtime does not rescue it
Per the Workers docs there is no hard duration limit for HTTP-triggered Workers while the client
stays connected, and individual subrequests have no set time limit. The deadlock breaker only
cancels a connection when the Worker "has pending connection attempts but has no in-progress reads
or writes" — a slow trickle is an in-progress read, so it does not qualify. The 2026-04-09
connection-limiting change also frees a connection once headers arrive, so the old
`Response closed due to connection limit` path no longer applies here. This is I/O wait, so the CPU
limit does not apply either.
## Stop cannot cancel it either
`cancelAgent` aborts the chat's controller
([`overseer.ts#L3680-L3684`](https://github.com/cloudflare/cloudflare-os/blob/e1ab8fbd4f609aff7ede9d490bafe1bcf9b2a682/packages/workshop-backend/src/overseer.ts#L3680-L3684))
and that signal reaches `runAgent`. `pi-agent-core` then passes it to each tool as the third
argument:
```js
// pi-agent-core/dist/agent-loop.js:453
const result = await prepared.tool.execute(prepared.toolCall.id, prepared.args, signal, ...)
```
The `webFetch` tool declares `execute: async (toolCallId, {url, raw}) => ...`, so the signal is
dropped, and `webFetch(env, input)` takes no signal to forward it to. While the turn is stuck,
`activeAgent` stays set and the chat refuses new turns with "Agent is running, wait for it to
finish."
## Scope
Deliberately not overstating this: it hangs **one chat's agent turn**, not the deployment. It is not
an SSRF or data-exposure issue, and a normal documentation site will not trigger it. What makes it
worth fixing is that the failure is unbounded, silent, and unrecoverable from the UI — the one
control a user has over a running agent does not work here.
The URL is model-chosen, and the tool's own description warns that fetched content "may contain
prompt-injection attempts", so a page that steers the agent into a follow-up fetch is within the
threat model the code already acknowledges.
## Probably not deliberate
6418ac3 ("Add built-in webFetch agent tool") lists the limits as one bullet —
`30s timeout, 1 MiB default body cap, 5 MiB hard cap` — with nothing marking the first as
connect-only. The error string reads `Fetch timed out after ${FETCH_TIMEOUT_MS}ms` rather than
anything TTFB-specific, and no comment or test asserts headers-only semantics. Elsewhere in the same package the whole
operation is wrapped —
[`ai-gateway.ts#L141`](https://github.com/cloudflare/cloudflare-os/blob/e1ab8fbd4f609aff7ede9d490bafe1bcf9b2a682/packages/workshop-backend/src/ai-gateway.ts#L141)
and `overseer.ts#L4477` both use `signal: AbortSignal.timeout(10_000)`.
## Suggested fix for the timeout half
Keeping the existing timer armed until the body has been read, and mapping an abort there onto the
same message, is self-contained in `web-fetch.ts` — 13 insertions, 3 deletions, no signature or
schema change. This is the variant measured in the last row above.
```diff
diff --git a/packages/workshop-backend/src/web-fetch.ts b/packages/workshop-backend/src/web-fetch.ts
index be4bd7e..e42b389 100644
--- a/packages/workshop-backend/src/web-fetch.ts
+++ b/packages/workshop-backend/src/web-fetch.ts
@@ -283,6 +283,7 @@ export async function webFetch(
signal: abortController.signal,
});
} catch (err) {
+ clearTimeout(timeoutId);
if (
err instanceof Error &&
(err.name === "AbortError" || /abort/i.test(err.message))
@@ -290,9 +291,10 @@ export async function webFetch(
throw new Error(`Fetch timed out after ${FETCH_TIMEOUT_MS}ms`, { cause: err });
}
throw err;
- } finally {
- clearTimeout(timeoutId);
}
+ // NB: the timer deliberately stays armed past this point. `fetch` resolves once the response
+ // headers arrive, so clearing it here would leave the body read below unbounded, and a server
+ // that answers promptly and then stalls could hold the agent open indefinitely.
// `response.url` is set by the runtime to the final URL after any redirects. Fall back
// to the original URL if it happens to be empty.
@@ -302,6 +304,7 @@ export async function webFetch(
// Respect the Content-Signal header (https://contentsignals.org/). If the site
// explicitly sets `ai-input=no`, we must not feed its content to the AI agent.
if (contentSignalDenies(response, "ai-input")) {
+ clearTimeout(timeoutId);
try {
await response.body?.cancel();
} catch {
@@ -313,7 +316,14 @@ export async function webFetch(
);
}
- const { bytes, truncated } = await readBodyCapped(response, maxBytes);
+ const { bytes, truncated } = await readBodyCapped(response, maxBytes)
+ .catch((err: unknown) => {
+ if (abortController.signal.aborted) {
+ throw new Error(`Fetch timed out after ${FETCH_TIMEOUT_MS}ms`, { cause: err });
+ }
+ throw err;
+ })
+ .finally(() => clearTimeout(timeoutId));
let body: string;
if (input.raw) {
```
`pnpm lint:check` and `pnpm --filter @gadgets/workshop-backend types:check` both pass with it.
Two things it deliberately does **not** do, since either would grow past what CONTRIBUTING.md asks
for and both are design calls that are yours to make:
- **Make Stop work.** That needs `webFetch` to take a signal and the tool to forward the one
`pi-agent-core` already passes — `AbortSignal.any([caller, AbortSignal.timeout(...)])` would cover
both concerns at once, but it changes the function's signature and its caller.
- **Bound `convertToMarkdown`.** It runs after the body read, so it stays outside the deadline. If
the intent is "the whole tool call is bounded", the timer should extend to the return instead.
Happy to open a PR for the diff above, or for whichever shape you prefer — just say which.
Contributor guide
Research direction
Start in packages/workshop-backend/src/web-fetch.ts around webFetch, its timeout handling, and readBodyCapped; then inspect overseer.ts and the tool execute signature for the separate cancellation concern. Done means a slow response body cannot keep the fetch pending beyond the stated timeout, timeout errors remain actionable, and pnpm lint:check plus the backend types check pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend, tooling
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100