structuredOutput(): a finish_reason=length truncation is reported as a JSON parse error (openai-base, ai-openrouter)
@AlemTuzlak is already working on this.
Since Sep 19, 2026.
- Dominant language
- TypeScript
- Stars
- 3.1k
- Forks
- 331
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 160
Description
TanStack AI version
@tanstack/ai 0.56.0, @tanstack/openai-base 0.10.13, @tanstack/ai-openrouter 0.19.13 (current main, bffdd186)
Framework/Library version
Any (server-side, framework-independent)
Describe the bug and the steps to reproduce it
The non-stream structuredOutput() in openai-base (chat-completions-text.ts) and ai-openrouter (text.ts) parses choices[0].message.content without looking at finish_reason. When the model hits the output cap, finish_reason is length, the content is a truncated JSON document, and JSON.parse fails — so the caller gets Failed to parse structured output as JSON. Content: …, a parse error, for what is a truncation.
Two consequences:
- A "raise
max_completion_tokens" problem is reported as "the model did not follow the schema". Anyone bucketing failures (schema mismatch vs. truncated vs. provider error) counts it in the wrong bucket, and the fix is invisible. - The parse-error message carries the first 200 characters of the model output, so the (possibly sensitive) raw text ends up in whatever logs the error message.
The chat stream path already distinguishes this: ai-anthropic emits RUN_ERROR with "The response was cut off because the maximum token limit was reached." on max_tokens (#849), and ai-bedrock maps max_tokens → finishReason: 'length'. The structured-output non-stream path in the OpenAI-shaped adapters has no equivalent check.
Steps to reproduce (no API key needed):
- Stub the SDK response as
{ choices: [{ message: { content: '{"title":"Hel' }, finish_reason: 'length' }] }. - Call
adapter.structuredOutput({ chatOptions, outputSchema }). - Observe: rejects with
Failed to parse structured output as JSON. Content: {"title":"Hel. Nothing in the error says the response was cut off.
Seen in practice with openRouterText on a free model with a modest maxCompletionTokens: the truncation surfaced as a schema failure in our evaluation report until we added a finish_reason check in front of the adapter.
Your Minimal, Reproducible Example - (Sandbox Highly Recommended)
Reproduces in the packages' own test suites with the stub above — the existing throws … Failed to parse structured output as JSON tests in packages/openai-base/tests/chat-completions-text.test.ts and packages/ai-openrouter/tests/openrouter-adapter.test.ts show the setup.
Screenshots or Videos (Optional)
No response
Do you intend to try to help solve this bug with your own PR?
Yes — PR incoming: check finish_reason === 'length' before parsing in both non-stream structuredOutput() implementations and throw a distinct truncation error (same wording as the Anthropic stream path), with tests and patch changesets. I'm leaving the raw-content-in-message question out of that PR; happy to follow up separately if you'd rather not have model output in error messages.
Terms & Code of Conduct
- I agree to follow this project's Code of Conduct
- I understand that if my bug cannot be reliable reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.