cloudflare / cloudflare/agents
Think schedules _chatRecoveryContinue for pre-first-chunk stalls, causing recovery to skip silently
- Dominant language
- TypeScript
- Stars
- 5.6k
- Forks
- 711
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 53
Description
(another ai generated issue but it properly explains the problem :) )
---
When a Think chat stream stalls before producing its first assistant chunk, the recovery system schedules `_chatRecoveryContinue`.
At this point there is no assistant message to continue from. The latest conversation leaf is still the user's message.
The scheduled callback eventually calls `continueLastTurn()`, which intentionally returns `status: "skipped"` when the latest leaf is not an assistant message. As a result, the original user turn is never retried and the recovery finishes silently.
This was observed with:
- `@cloudflare/think@0.12.1`
- Chat recovery enabled
- A stream stall before the first assistant part
- The current `@cloudflare/think@0.13.0` source appears to contain the same behavior
## Reproduction
1. Create a Think chat agent with chat recovery enabled.
2. Configure a relatively short `chatStreamStallTimeoutMs`.
3. Use a model/provider that accepts the request but does not produce a first stream chunk before the timeout.
4. Send a user message.
5. Allow the watchdog to detect the stall.
6. Observe the scheduled `_chatRecoveryContinue` callback.
## Actual behavior
The live stream stall follows this path:
```text
ChatStreamStalledError
-> _routeStallToBoundedRecovery()
-> schedule _chatRecoveryContinue
-> close the live stream with an empty done response
-> _chatRecoveryContinue()
-> continueLastTurn()
-> latest leaf is the user message
-> status: "skipped"
```
`_routeStallToBoundedRecovery()` always creates the incident with:
```ts
recoveryKind: "continue"
```
and always schedules:
```ts
callback: "_chatRecoveryContinue"
```
even when `targetAssistantId` is absent because no assistant content was produced.
`continueLastTurn()` then returns early:
```ts
if (!lastLeaf || lastLeaf.role !== "assistant") {
return { requestId: "", status: "skipped" }
}
```
The incident is updated to `skipped`, but the original user turn is not retried.
## Lifecycle hook gap
This path also does not invoke any terminal application lifecycle hook:
- `onChatRecovery()` is invoked by fiber recovery, but not when this live-stall callback is scheduled.
- `onChatResponse()` is not called because no continuation turn starts.
- `onChatError()` is not called because the stall was routed into recovery.
- `onExhausted()` is not called because `skipped` is not treated as exhaustion.
This prevents applications from reliably cleaning up turn-scoped resources such as telemetry spans, metrics, timers, or custom state.
The client receives a clean `done: true` response with an empty body, so the visible result may simply be a silent turn with no assistant response or terminal recovery message.
## Expected behavior
A stall before the first assistant chunk should retry the last user turn rather than trying to continue a nonexistent assistant message.
One possible strategy:
```ts
const hasPartialAssistant = Boolean(input.targetAssistantId)
const recoveryKind = hasPartialAssistant ? "continue" : "retry"
const callback = hasPartialAssistant
? "_chatRecoveryContinue"
: "_chatRecoveryRetry"
```
The retry callback could use the latest user message ID already resolved by `_routeStallToBoundedRecovery()`.
Additionally, every scheduled recovery should eventually produce an application-visible terminal lifecycle event, including skipped recoveries.
Possible API options:
- Invoke `onChatRecovery()` when live-stall recovery is scheduled.
- Add `onChatRecoverySkipped()`.
- Invoke `onChatResponse()` with `status: "skipped"`.
- Route an unrecoverable skip through `onExhausted()` and the configured terminal message.
## Impact
- The user's original message is never retried.
- The chat can end silently without an assistant response.
- The configured terminal recovery message is not delivered.
- Turn-scoped application resources can remain open until another turn starts.
- Telemetry may show an unfinished turn despite the recovery callback having completed.
Contributor guide
Assessment
This issue has not been assessed yet.