cloudflare / cloudflare/agents

[think] Tool outputs are silently truncated to 500 chars

Open
#2,014 2 comments 0 reactions 1 assignee Claimed by @threepointone View on GitHub
bug think
Dominant language
TypeScript
Stars
5.6k
Forks
711
Avg merge
1d 20h
Merged PRs (30d)
53

Description

ai slop but still true:

**Describe the bug**

`Think._assembleModelMessages` calls `truncateOlderMessages` unconditionally:

```js
// @cloudflare/think@0.15.1 dist/think.js:2463
const truncated = truncateOlderMessages(await this._repairTranscriptForProvider(this.messages));
```

No options are passed, so the defaults bind: `keepRecent: 4`, `maxToolOutputChars: 500`.

For oversized structured outputs, `truncateToolOutput` preserves the container shape but replaces the contents with marker keys:

```js
// agents@0.20.1 dist/tool-output-truncation-CNnnGZQ3.js:3-4
const TRUNCATED_FLAG = "__truncated";
const TRUNCATED_CHARS = "__truncatedChars";
```

Those markers do not satisfy the tool's declared output schema. Any `toModelOutput` that validates its input — for example with a zod `.parse()` — throws.

The failure is permanent rather than transient, because the AI SDK invokes `toModelOutput` from inside `convertToModelMessages`, which re-runs on **every replay of the thread**. So once a tool output ages past the 4-message window, every subsequent turn throws the same error at assembly time, the model request is never sent, and the thread is unusable until history is cleared.

**To Reproduce**

Steps to reproduce the behavior:

1. Build a Think agent with a tool whose `toModelOutput` validates the persisted output against its declared schema (a throwing `.parse`).
2. Call that tool so it returns a structured output larger than 500 characters.
3. Send five or more further turns, pushing the tool call out of the `keepRecent: 4` window.
4. Send one more message.
5. See validation fail on the `{__truncated, __truncatedChars}` markers. Every subsequent turn fails identically.

**Expected behavior**

Truncation should not produce tool outputs that violate the schema the tool itself declares. Options:

- exclude structured outputs with a declared schema from truncation, or
- expose the truncation options on `Think` (`keepRecent`, `maxToolOutputChars`, `maxTextChars`, including opting out), or
- at minimum, make a truncation-induced validation failure non-fatal so one aged tool output cannot wedge a thread permanently.

**Screenshots**

N/A.

**Version:**

`@cloudflare/think@0.15.1` with `agents@0.20.1`.

**Additional context**

This is the same failure mode as #1498, which was closed by making Think's own `read.toModelOutput` tolerant of truncated input. The truncation side was not changed — it now preserves the container shape instead of stringifying the whole output, but the contents still violate strict schemas, so any tool that validates hits it again.

We currently mitigate twice: `safeParse` with a raw-output fallback in our own `toModelOutput`, and a patch removing the `truncateOlderMessages` call from `_assembleModelMessages`. The first costs us citations on truncated turns; the second is a fork of the SDK.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.