MoonshotAI / MoonshotAI/kimi-code

[v2 engine] No capability-driven media degradation: pasted images to text-only models fail with provider 400 (unknown variant 'image_url')

Open
#2,954 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
7.5k
Forks
1.2k
Avg merge
11h 53m
Merged PRs (30d)
350

Description

Summary / 摘要

The v2 engine (agent-core-v2, default since 0.33.0) does not degrade media parts based on the active model's capabilities before sending a request. A pasted image sent to a text-only model through a third-party OpenAI-compatible provider fails with a provider 400 (unknown variant 'image_url', expected 'text'), even though the v1 (legacy) engine handles the same scenario by replacing image_url parts with a text placeholder.

v2 引擎(0.33.0 起默认的 agent-core-v2)在发送请求前不会根据模型能力对媒体块做降级。通过第三方 OpenAI 兼容网关向纯文本模型粘贴图片时,provider 会返回 400(unknown variant 'image_url', expected 'text');而 v1(legacy)引擎能正确处理同一场景——在请求前把 image_url 块替换成文本占位符。

Environment / 环境

  • Kimi Code 0.36.1
  • Provider: custom OpenAI-compatible endpoint (opencode zen gateway) with a text-only model (deepseek-v4-flash)
  • 自定义 OpenAI 兼容端点 + 纯文本模型(deepseek-v4-flash)

Repro / 复现步骤

  1. In config.toml, declare a model whose top-level capabilities include image_in (the TUI paste gate reads the raw top-level capabilities), but whose [models."...".overrides] capability list omits image_in (so the engine's effectiveModelAlias sees no image support):
[models."opencode-go/deepseek-v4-flash"]
capabilities = [ "image_in", "always_thinking", "tool_use" ]

[models."opencode-go/deepseek-v4-flash".overrides]
capabilities = [ "always_thinking", "tool_use" ]
  1. Paste an image (Alt-V) and submit. 粘贴图片并提交。
  2. The TUI gate passes (top-level image_in), the message carries an image_url part, and the v2 engine sends it verbatim to the provider → HTTP 400 unknown variant 'image_url', expected 'text'.

With KIMI_CODE_LEGACY_FLAG=1 (v1 engine) the same setup works: downgradeUnsupportedMedia (packages/agent-core/src/agent/turn/kosong-llm.ts) replaces the image_url part with [image omitted: current model has no image input] before the request goes out.

Root cause / 根因

  • The TUI paste gate (apps/kimi-code/src/tui/kimi-tui.tsvalidateMediaCapabilities / supportsCurrentModelCapability) checks availableModels[model].capabilities, which comes from the raw top-level config.models entry, not from effectiveModelAlias. So a top-level image_in declaration is what unlocks pasting. (TUI 粘贴检查读的是 config.toml 顶层 capabilities,而非合并 overrides 后的 effectiveModelAlias。)
  • The v2 engine (packages/agent-core-v2) has no capability-driven media degradation. Its only media fallbacks are projectMediaDegraded / projectMediaStripped (packages/agent-core-v2/src/agent/contextProjector/contextProjectorService.ts), which are recovery projections enabled only after a request failure, and they keep the most recent 2 media parts (MEDIA_DEGRADE_KEEP_RECENT = 2) — so a freshly pasted image is never degraded and the retry fails with the same 400. (v2 只有失败后重试时的恢复投影,且保留最近 2 条媒体,刚粘贴的图片永远不会被降级。)
  • The v1 engine (packages/agent-core) degrades media before the request based on capability.image_in (kosong-llm.ts downgradeUnsupportedMedia). (v1 在请求前按能力主动降级。)

Use case / 使用场景

Users running text-only models (e.g. DeepSeek V4 Flash via a third-party OpenAI-compatible gateway) want to paste images and have a local vision bridge — for example a UserPromptSubmit hook that converts the image into text evidence via a separate vision engine — handle them. For that flow to work, the client must (a) let the paste through and (b) not send the raw image_url to a provider that rejects it. With the v2 engine only (a) is possible via the capability declaration; (b) has no equivalent of v1's capability-driven degradation.

纯文本模型(如经第三方 OpenAI 兼容网关的 deepseek-v4-flash)+ 本地视觉桥接(例如用 UserPromptSubmit hook 调用外部视觉引擎把图片转成文本证据)的场景需要:客户端放行粘贴,同时不把 image_url 发给拒绝它的 provider。v2 下只能做到前者,后者没有 v1 那样的能力降级机制。

Suggested fix / 建议

  1. In v2, add request-time media degradation driven by the model's effective capabilities, mirroring v1's downgradeUnsupportedMedia: when the effective capability lacks image_in, replace image_url parts with a text placeholder before the request (reuse or extend the existing MEDIA_STRIPPED_PLACEHOLDERS text). (v2 增加基于有效能力的请求前媒体降级,对齐 v1。)
  2. Alternatively/additionally, make the TUI paste gate (validateMediaCapabilities) use effectiveModelAlias instead of the raw top-level capabilities, so a top-level image_in declaration can no longer diverge from what the engine actually sees — the current override workaround would then be either fully supported or clearly rejected at paste time. (或让 TUI 粘贴检查改用 effectiveModelAlias,消除顶层/overrides 能力不一致。)
  3. Consider exposing whether media was degraded in the request log / telemetry (v2 already tracks mediaDegradedTurns / mediaStrippedTurns). (考虑在请求日志/遥测中暴露媒体是否被降级。)

Thanks for considering. Happy to provide a full repro config or more detail.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with packages/agent-core-v2 and compare its request path with downgradeUnsupportedMedia in packages/agent-core/src/agent/turn/kosong-llm.ts. Review effectiveModelAlias, contextProjectorService.ts, and the TUI checks in apps/kimi-code/src/tui/kimi-tui.ts. Reproduce the configuration in the issue, then verify that a text-only model no longer receives image_url and the request succeeds without the provider 400.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.