TanStack / TanStack/ai

structuredOutput(): a finish_reason=length truncation is reported as a JSON parse error (openai-base, ai-openrouter)

Open
#1,426 1 comment 0 reactions 1 assignee View on GitHub

@AlemTuzlak is already working on this.

Since Sep 19, 2026.

has-pr needs-repro waiting-on: maintainer
Dominant language
TypeScript
Stars
3.1k
Forks
331
Avg merge
1d 22h
Merged PRs (30d)
160

Description

TanStack AI version

@tanstack/ai 0.56.0, @tanstack/openai-base 0.10.13, @tanstack/ai-openrouter 0.19.13 (current main, bffdd186)

Framework/Library version

Any (server-side, framework-independent)

Describe the bug and the steps to reproduce it

The non-stream structuredOutput() in openai-base (chat-completions-text.ts) and ai-openrouter (text.ts) parses choices[0].message.content without looking at finish_reason. When the model hits the output cap, finish_reason is length, the content is a truncated JSON document, and JSON.parse fails — so the caller gets Failed to parse structured output as JSON. Content: …, a parse error, for what is a truncation.

Two consequences:

  1. A "raise max_completion_tokens" problem is reported as "the model did not follow the schema". Anyone bucketing failures (schema mismatch vs. truncated vs. provider error) counts it in the wrong bucket, and the fix is invisible.
  2. The parse-error message carries the first 200 characters of the model output, so the (possibly sensitive) raw text ends up in whatever logs the error message.

The chat stream path already distinguishes this: ai-anthropic emits RUN_ERROR with "The response was cut off because the maximum token limit was reached." on max_tokens (#849), and ai-bedrock maps max_tokensfinishReason: 'length'. The structured-output non-stream path in the OpenAI-shaped adapters has no equivalent check.

Steps to reproduce (no API key needed):

  1. Stub the SDK response as { choices: [{ message: { content: '{"title":"Hel' }, finish_reason: 'length' }] }.
  2. Call adapter.structuredOutput({ chatOptions, outputSchema }).
  3. Observe: rejects with Failed to parse structured output as JSON. Content: {"title":"Hel. Nothing in the error says the response was cut off.

Seen in practice with openRouterText on a free model with a modest maxCompletionTokens: the truncation surfaced as a schema failure in our evaluation report until we added a finish_reason check in front of the adapter.

Your Minimal, Reproducible Example - (Sandbox Highly Recommended)

Reproduces in the packages' own test suites with the stub above — the existing throws … Failed to parse structured output as JSON tests in packages/openai-base/tests/chat-completions-text.test.ts and packages/ai-openrouter/tests/openrouter-adapter.test.ts show the setup.

Screenshots or Videos (Optional)

No response

Do you intend to try to help solve this bug with your own PR?

Yes — PR incoming: check finish_reason === 'length' before parsing in both non-stream structuredOutput() implementations and throw a distinct truncation error (same wording as the Anthropic stream path), with tests and patch changesets. I'm leaving the raw-content-in-message question out of that PR; happy to follow up separately if you'd rather not have model output in error messages.

Terms & Code of Conduct
  • I agree to follow this project's Code of Conduct
  • I understand that if my bug cannot be reliable reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.