TanStack / TanStack/ai

openaiCompatible: summarize({ maxLength }) sends no token cap

Open
#1,385 0 comments 0 reactions 1 assignee View on GitHub

@jherr is already working on this.

Since Sep 17, 2026.

bug has-pr waiting-on: maintainer
Dominant language
TypeScript
Stars
3.1k
Forks
331
Avg merge
1d 22h
Merged PRs (30d)
160

Description

TanStack AI version

@tanstack/ai 0.54.0, @tanstack/ai-openai 0.22.6 (current main)

Framework/Library version

Any (server-side, framework-independent)

Describe the bug and the steps to reproduce it

summarize({ maxLength }) sends no output token cap for any provider reached through openaiCompatible (DeepSeek, Moonshot/Kimi, Together, Fireworks, Qwen, vLLM, and others).

openaiCompatible has no summarize factory. To summarize, you wrap the text adapter in ChatStreamSummarizeAdapter. That wrapper chooses the token key from its own name, using MAX_TOKENS_KEY_BY_ADAPTER in packages/ai/src/activities/summarize/chat-stream-summarize.ts. That map only has first-party names (openai, anthropic, groq, llmgateway, cloudflare, ...). A compatible provider's name is chosen by the user, so it is never in the map:

  • With the default name chat-stream-summarize, the cap is dropped.
  • With the provider name, for example deepseek, the cap is dropped.
  • With openai, the key is max_output_tokens. That is the Responses API key, and the Chat Completions endpoint ignores it.

In each case, the SDK logs a warning and sends the request without max_tokens or max_completion_tokens. Only the prompt asks the model to stay short.

Steps:

import { summarize, ChatStreamSummarizeAdapter } from '@tanstack/ai'
import { openaiCompatible } from '@tanstack/ai-openai/compatible'

const deepseek = openaiCompatible({
  name: 'deepseek',
  baseURL: 'https://api.deepseek.com/v1',
  apiKey: process.env.DEEPSEEK_API_KEY ?? '',
  models: ['deepseek-chat'],
})

const adapter = new ChatStreamSummarizeAdapter(
  deepseek('deepseek-chat'),
  'deepseek-chat',
  'deepseek',
)

await summarize({ adapter, text: longText, maxLength: 100 })

Expected: the Chat Completions request body contains an output cap (max_tokens or max_completion_tokens).

Actual: the request body has no output cap. The logger warns maxLength=100 could not be mapped to a provider token key for adapter name "deepseek".

Possible fixes:

  1. Choose the key from the wrapped adapter's wire API, not from a name. Every Chat Completions adapter on OpenAIBaseChatCompletionsTextAdapter would get a Chat Completions key. This also removes the need for per-gateway entries such as llmgateway and cloudflare.
  2. Add a summarize factory to openaiCompatible, with an optional maxTokensKey setting.

Related: the other openaiCompatible gap, reasoning deltas (delta.reasoning_content / delta.reasoning) that are never surfaced, is tracked in #982 and fixed in #1367.

Your Minimal, Reproducible Example - (Sandbox Highly Recommended)

See the snippet above. Code path: ChatStreamSummarizeAdapter.buildTextOptions, then isKnownMaxTokensAdapter(this.name) and applyMaxLength.

Do you intend to try to help solve this bug with your own PR?

Maybe, I'll investigate and start debugging

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.