cloudflare / cloudflare/agents

Think terminalizes transient provider/gateway stream errors instead of routing them through bounded chatRecovery — classifyChatError "transient" is a no-op

Open
#2,085 0 comments 0 reactions 1 assignee Claimed by @threepointone View on GitHub
enhancement think
Dominant language
TypeScript
Stars
5.6k
Forks
711
Avg merge
1d 20h
Merged PRs (30d)
53

Description

**Describe the bug**

When a provider/gateway stream terminates with an error mid-turn (the upstream ends the SSE stream before the terminal chunk), Think terminalizes the turn instead of routing it through the bounded `chatRecovery` machinery — even though `ChatErrorClassification` already reserves `"transient"` and `"rate_limit"` for exactly this, and the app's `classifyChatError` hook could classify it.

In the stream catch path, only `ChatStreamStalledError` is routed into `_routeStallToBoundedRecovery`. Every other stream error falls through to the generic terminal path: the resumable stream is errored, a terminal error frame is broadcast, `onChatError` fires, and the turn dies. The `classifyChatError` docs confirm the hook "currently drives **only** context-overflow recovery" and that the other categories are "reserved for future use — returning one today is a no-op."

This produces a perverse asymmetry: if the upstream had **hung silently**, the stream-stall watchdog would have fired and the turn would have been recovered via the bounded-recovery route. Because the upstream instead **failed fast with a clean error**, the turn got worse treatment — it terminalized with no retry, leaving a dangling mid-flight tool call (later repaired to `output-error` with "The tool call was interrupted before a result was recorded") and no final assistant text.

**To Reproduce**

Steps to reproduce the behavior:

1. Run a `Think` (or `AIChatAgent`) turn with `chatRecovery` enabled and a multi-step tool loop in progress.
2. Have the model provider (or an AI gateway in front of it) terminate the response stream before the terminal/finish chunk — e.g. proxy the provider through something that kills the upstream connection mid-generation, ideally while a tool call's input is still streaming.
3. Observe the turn terminalize: terminal error frame broadcast, `onChatError` invoked with the stream error, no recovery incident opened, no continuation scheduled.
4. Compare with the stall case: make the same upstream *hang* (stay open, send nothing) past `chatStreamStallTimeoutMs` instead — the watchdog throws `ChatStreamStalledError` and the turn *is* recovered via the bounded route.
5. Note that overriding `classifyChatError` to return `"transient"` for the error in step 2 changes nothing (documented no-op).

**Expected behavior**

When a stream error occurs and the app's `classifyChatError` returns `"transient"` (or `"rate_limit"`), route the interruption through the same bounded recovery path the stall watchdog already uses (`_routeStallToBoundedRecovery`-style: open/advance an incident, respect `maxAttempts` + wall-clock window + progress-aware budget resets, schedule a continuation, deliver the configured terminal UX only on exhaustion).

Everything needed already exists — incident dedup, attempt budgets, progress markers, and the transcript-repair idempotency story (settled tool results are durable; the interrupted mid-flight tool call is repaired to a settled error, so only that call re-runs on continuation). This is plumbing the reserved classification categories into an existing route, not new machinery. Errors classified `"fatal"`/`"unknown"` (or when the hook is not overridden) should keep today's terminal behavior.

**Screenshots**

N/A — server-side. The failure mode is the provider (or a gateway proxying it) ending the response stream with a structured "upstream ended before the terminal/finish chunk" error while a step is mid-flight; no usage is reported for the interrupted call.

**Version:**

- `@cloudflare/think@0.15.1`
- `agents@0.20.1`
- `ai@6.0.208`

**Additional context**

- Sibling issue: #1964 asks for the same catch-path gap to be fixed for **Cloudflare platform transients** (errors `isPlatformTransientError` already recognizes, e.g. `Network connection lost.`). Its expected behavior deliberately leaves provider errors terminal. This issue is the complement: provider/gateway stream terminations can't be recognized by the package's platform classifier, but the app *can* classify them — Think just doesn't act on the classification. Fixing #1964 alone would still leave this case terminal; honoring `classifyChatError` covers both with one route.
- #1626 (closed) established the stall → bounded-recovery route that this would generalize.
- Request-level retries don't help: AI SDK `maxRetries` retries failed *calls*, and gateway-level retry/fallback is request-scoped — once tokens have started flowing, a mid-stream provider drop is non-resumable and the step has to be restarted, which is exactly what bounded recovery does.
- A secondary observability nit (also noted in #1964): the generic catch flattens the error to a string before `onChatError` / `chat:request:failed`, discarding the structured payload (code, cause), which makes app-side classification of exactly these errors harder than it needs to be.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.