MoonshotAI / MoonshotAI/kimi-code
[v2 engine] No capability-driven media degradation: pasted images to text-only models fail with provider 400 (unknown variant 'image_url')
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 7.5k
- Forks
- 1.2k
- Avg merge
- 11h 53m
- Merged PRs (30d)
- 350
Description
Summary / 摘要
The v2 engine (agent-core-v2, default since 0.33.0) does not degrade media parts based on the active model's capabilities before sending a request. A pasted image sent to a text-only model through a third-party OpenAI-compatible provider fails with a provider 400 (unknown variant 'image_url', expected 'text'), even though the v1 (legacy) engine handles the same scenario by replacing image_url parts with a text placeholder.
v2 引擎(0.33.0 起默认的 agent-core-v2)在发送请求前不会根据模型能力对媒体块做降级。通过第三方 OpenAI 兼容网关向纯文本模型粘贴图片时,provider 会返回 400(unknown variant 'image_url', expected 'text');而 v1(legacy)引擎能正确处理同一场景——在请求前把 image_url 块替换成文本占位符。
Environment / 环境
- Kimi Code 0.36.1
- Provider: custom OpenAI-compatible endpoint (opencode zen gateway) with a text-only model (
deepseek-v4-flash) - 自定义 OpenAI 兼容端点 + 纯文本模型(deepseek-v4-flash)
Repro / 复现步骤
- In
config.toml, declare a model whose top-levelcapabilitiesincludeimage_in(the TUI paste gate reads the raw top-level capabilities), but whose[models."...".overrides]capability list omitsimage_in(so the engine'seffectiveModelAliassees no image support):
[models."opencode-go/deepseek-v4-flash"]
capabilities = [ "image_in", "always_thinking", "tool_use" ]
[models."opencode-go/deepseek-v4-flash".overrides]
capabilities = [ "always_thinking", "tool_use" ]
- Paste an image (
Alt-V) and submit. 粘贴图片并提交。 - The TUI gate passes (top-level
image_in), the message carries animage_urlpart, and the v2 engine sends it verbatim to the provider → HTTP 400unknown variant 'image_url', expected 'text'.
With KIMI_CODE_LEGACY_FLAG=1 (v1 engine) the same setup works: downgradeUnsupportedMedia (packages/agent-core/src/agent/turn/kosong-llm.ts) replaces the image_url part with [image omitted: current model has no image input] before the request goes out.
Root cause / 根因
- The TUI paste gate (
apps/kimi-code/src/tui/kimi-tui.ts—validateMediaCapabilities/supportsCurrentModelCapability) checksavailableModels[model].capabilities, which comes from the raw top-levelconfig.modelsentry, not fromeffectiveModelAlias. So a top-levelimage_indeclaration is what unlocks pasting. (TUI 粘贴检查读的是 config.toml 顶层 capabilities,而非合并 overrides 后的 effectiveModelAlias。) - The v2 engine (
packages/agent-core-v2) has no capability-driven media degradation. Its only media fallbacks areprojectMediaDegraded/projectMediaStripped(packages/agent-core-v2/src/agent/contextProjector/contextProjectorService.ts), which are recovery projections enabled only after a request failure, and they keep the most recent 2 media parts (MEDIA_DEGRADE_KEEP_RECENT = 2) — so a freshly pasted image is never degraded and the retry fails with the same 400. (v2 只有失败后重试时的恢复投影,且保留最近 2 条媒体,刚粘贴的图片永远不会被降级。) - The v1 engine (
packages/agent-core) degrades media before the request based oncapability.image_in(kosong-llm.tsdowngradeUnsupportedMedia). (v1 在请求前按能力主动降级。)
Use case / 使用场景
Users running text-only models (e.g. DeepSeek V4 Flash via a third-party OpenAI-compatible gateway) want to paste images and have a local vision bridge — for example a UserPromptSubmit hook that converts the image into text evidence via a separate vision engine — handle them. For that flow to work, the client must (a) let the paste through and (b) not send the raw image_url to a provider that rejects it. With the v2 engine only (a) is possible via the capability declaration; (b) has no equivalent of v1's capability-driven degradation.
纯文本模型(如经第三方 OpenAI 兼容网关的 deepseek-v4-flash)+ 本地视觉桥接(例如用 UserPromptSubmit hook 调用外部视觉引擎把图片转成文本证据)的场景需要:客户端放行粘贴,同时不把 image_url 发给拒绝它的 provider。v2 下只能做到前者,后者没有 v1 那样的能力降级机制。
Suggested fix / 建议
- In v2, add request-time media degradation driven by the model's effective capabilities, mirroring v1's
downgradeUnsupportedMedia: when the effective capability lacksimage_in, replaceimage_urlparts with a text placeholder before the request (reuse or extend the existingMEDIA_STRIPPED_PLACEHOLDERStext). (v2 增加基于有效能力的请求前媒体降级,对齐 v1。) - Alternatively/additionally, make the TUI paste gate (
validateMediaCapabilities) useeffectiveModelAliasinstead of the raw top-level capabilities, so a top-levelimage_indeclaration can no longer diverge from what the engine actually sees — the current override workaround would then be either fully supported or clearly rejected at paste time. (或让 TUI 粘贴检查改用 effectiveModelAlias,消除顶层/overrides 能力不一致。) - Consider exposing whether media was degraded in the request log / telemetry (v2 already tracks
mediaDegradedTurns/mediaStrippedTurns). (考虑在请求日志/遥测中暴露媒体是否被降级。)
Thanks for considering. Happy to provide a full repro config or more detail.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with packages/agent-core-v2 and compare its request path with downgradeUnsupportedMedia in packages/agent-core/src/agent/turn/kosong-llm.ts. Review effectiveModelAlias, contextProjectorService.ts, and the TUI checks in apps/kimi-code/src/tui/kimi-tui.ts. Reproduce the configuration in the issue, then verify that a text-only model no longer receives image_url and the request succeeds without the provider 400.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100