MoonshotAI / MoonshotAI/kimi-code

FALLBACK_MAX_TOKENS=128000 rejected by Anthropic-compatible providers with lower output limits

Open
#2,062 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
7.5k
Forks
1.2k
Avg merge
11h 53m
Merged PRs (30d)
350

Description

Problem

When using an Anthropic-compatible provider (e.g., Volcano Engine) with a non-Claude model like k2.7, these providers expose an Anthropic-compatible interface with a max_tokens cap of 32768. However, kosong falls back to FALLBACK_MAX_TOKENS = 128000 for non-Claude models, which far exceeds the actual limit, causing an immediate 400 rejection:

400 {"error":{"code":"InvalidParameter","message":"The parameter max_tokens specified in the request is not valid: integer above maximum value, expected a value <= 32768, but got 128000 instead."}}

Root Cause

resolveDefaultMaxTokens() in packages/kosong/src/providers/anthropic.ts:257 only recognizes Claude model names. For non-Claude models it returns FALLBACK_MAX_TOKENS = 128000 without considering the provider's baseUrl. The same issue exists in the v2 copy at packages/agent-core-v2/src/kosong/provider/bases/anthropic/anthropic.ts.

Proposed Fix

1. Conservative fallback

When baseUrl points to a custom endpoint and the model isn't a recognized Claude, use 32768 (the most common output limit among Anthropic-compatible providers) instead of 128000.

2. Error-driven recovery

On 400 errors mentioning a max_tokens limit, parse the actual limit from the error message, clamp via withMaxCompletionTokens(), and retry. Follows the same pattern as the existing context-overflow / image-format / tool-adjacency recovery in errors.ts.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with resolveDefaultMaxTokens() in packages/kosong/src/providers/anthropic.ts:257 and compare it with the v2 copy at packages/agent-core-v2/src/kosong/provider/bases/anthropic/anthropic.ts. Then read the existing recovery logic in errors.ts, especially the context-overflow, image-format, and tool-adjacency handling. Done means custom Anthropic-compatible endpoints avoid the excessive fallback and max_tokens-limit errors can recover using the provider’s reported limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.