cloudflare / cloudflare/agents

Think schedules _chatRecoveryContinue for pre-first-chunk stalls, causing recovery to skip silently

Open
#1,941 0 comments 0 reactions 1 assignee Claimed by @threepointone View on GitHub
bug think
Dominant language
TypeScript
Stars
5.6k
Forks
711
Avg merge
1d 20h
Merged PRs (30d)
53

Description

(another ai generated issue but it properly explains the problem :) )

---

When a Think chat stream stalls before producing its first assistant chunk, the recovery system schedules `_chatRecoveryContinue`.

At this point there is no assistant message to continue from. The latest conversation leaf is still the user's message.

The scheduled callback eventually calls `continueLastTurn()`, which intentionally returns `status: "skipped"` when the latest leaf is not an assistant message. As a result, the original user turn is never retried and the recovery finishes silently.

This was observed with:

- `@cloudflare/think@0.12.1`
- Chat recovery enabled
- A stream stall before the first assistant part
- The current `@cloudflare/think@0.13.0` source appears to contain the same behavior

## Reproduction

1. Create a Think chat agent with chat recovery enabled.
2. Configure a relatively short `chatStreamStallTimeoutMs`.
3. Use a model/provider that accepts the request but does not produce a first stream chunk before the timeout.
4. Send a user message.
5. Allow the watchdog to detect the stall.
6. Observe the scheduled `_chatRecoveryContinue` callback.

## Actual behavior

The live stream stall follows this path:

```text
ChatStreamStalledError
-> _routeStallToBoundedRecovery()
-> schedule _chatRecoveryContinue
-> close the live stream with an empty done response
-> _chatRecoveryContinue()
-> continueLastTurn()
-> latest leaf is the user message
-> status: "skipped"
```

`_routeStallToBoundedRecovery()` always creates the incident with:

```ts
recoveryKind: "continue"
```

and always schedules:

```ts
callback: "_chatRecoveryContinue"
```

even when `targetAssistantId` is absent because no assistant content was produced.

`continueLastTurn()` then returns early:

```ts
if (!lastLeaf || lastLeaf.role !== "assistant") {
return { requestId: "", status: "skipped" }
}
```

The incident is updated to `skipped`, but the original user turn is not retried.

## Lifecycle hook gap

This path also does not invoke any terminal application lifecycle hook:

- `onChatRecovery()` is invoked by fiber recovery, but not when this live-stall callback is scheduled.
- `onChatResponse()` is not called because no continuation turn starts.
- `onChatError()` is not called because the stall was routed into recovery.
- `onExhausted()` is not called because `skipped` is not treated as exhaustion.

This prevents applications from reliably cleaning up turn-scoped resources such as telemetry spans, metrics, timers, or custom state.

The client receives a clean `done: true` response with an empty body, so the visible result may simply be a silent turn with no assistant response or terminal recovery message.

## Expected behavior

A stall before the first assistant chunk should retry the last user turn rather than trying to continue a nonexistent assistant message.

One possible strategy:

```ts
const hasPartialAssistant = Boolean(input.targetAssistantId)

const recoveryKind = hasPartialAssistant ? "continue" : "retry"
const callback = hasPartialAssistant
? "_chatRecoveryContinue"
: "_chatRecoveryRetry"
```

The retry callback could use the latest user message ID already resolved by `_routeStallToBoundedRecovery()`.

Additionally, every scheduled recovery should eventually produce an application-visible terminal lifecycle event, including skipped recoveries.

Possible API options:

- Invoke `onChatRecovery()` when live-stall recovery is scheduled.
- Add `onChatRecoverySkipped()`.
- Invoke `onChatResponse()` with `status: "skipped"`.
- Route an unrecoverable skip through `onExhausted()` and the configured terminal message.

## Impact

- The user's original message is never retried.
- The chat can end silently without an assistant response.
- The configured terminal recovery message is not delivered.
- Turn-scoped application resources can remain open until another turn starts.
- Telemetry may show an unfinished turn despite the recovery callback having completed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.