zai-org / zai-org/feedback

带图请求未按模型 modalities 校验,对不支持图片的模型仍发送 image_url 导致上游 400

Open
#570 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

priority: P2
Dominant language
No language data
Stars
22
Forks
1
PR merge metrics
No merged PRs in 30d

Description

问题

ZCode 在请求体中携带图片(image_url)时,没有检查当前选定模型的 modalities.input 是否包含 image。当用户切到一个只声明 input: ["text"] 的模型后,图片内容仍被原样塞入请求体发送,上游(非视觉模型)直接返回 400 InvalidParameter,整轮对话中断。

复现

  1. 在 provider(本例为自建 ai-gateway 反代,OpenAI-compatible 协议)下配置两个模型:
    • api/deepseek-v4-promodalities.input = ["text"](纯文本)
    • api/kimi-k2.6modalities.input = ["text", "image"](支持图片)
  2. 将当前模型切换到 api/deepseek-v4-pro
  3. 在对话中使用能产生图片内容的操作(如粘贴截图、Read 工具读取 PNG 文件)
  4. ZCode 将带 image_url 块的请求体发给 api/deepseek-v4-pro → 上游返回 400

报错信息

Model do not support image input.
Request id: 021788846573420402c3547e09b7409e300241a04980e228ce7ce
Turn execution failed
provider=ef85d547-... model=api/deepseek-v4-pro
reason=invalid_request status=400 retryable=false

期望

请求构建阶段读取当前模型的 modalities.input,如果包含 image 才允许携带图片内容;如果当前模型不支持图片而上下文中存在图片,应该:

  • 至少在发送前拦截并提示用户切换到支持图片的模型(同 provider 下有 modalities.input 包含 image 的模型可选时给出建议)
  • 或自动降级到同 provider 下支持图片的模型

而不是把请求原样发给上游、等一个必然的 400 再中断对话。

环境信息

  • ZCode CLI(macOS arm64)
  • provider: 自建 OpenAI-compatible 反代(ai-gateway),模型 modalities 在 ~/.zcode/v2/config.json 中正确声明(api/deepseek-v4-promodalities.input 只有 ["text"],不含 image

根因判断

ZCode 的 v2/config.json 中每个模型都有 modalities 字段,但请求构建流程似乎没有在拼装 image_url 之前校验这个字段。元数据声明了不支持图片,却仍然传了图片。

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the ZCode request-building flow and how model metadata is loaded from ~/.zcode/v2/config.json. Reproduce with one text-only model and one model whose modalities.input includes image, then verify that image content is intercepted before sending to the text-only model and that the user receives the intended guidance or fallback behavior.

Written by the indexing model from the issue text.

Assessment

Domain
api, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.