makecindy / makecindy/cindy

openai-chat 桥接因硬编码白名单丢弃图片输入,第三方兼容网关一律无法传图

Open
#2,371 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
2.7k
Forks
395
Avg merge
21h 48m
Merged PRs (30d)
776

Description

**提交人**: SwaYing
**客户端版本**: 0.1.38

---

## 现象

使用 `wireProtocol: "openai-chat"` 的自定义供应商时,发送图片必然失败,与下游模型是否多模态无关。

```
{"error":{"type":"invalid_request_error","code":"unsupported_feature",
"message":"Responses feature is not supported by the Chat Completions bridge: input content part 'input_image'"}}
```

本机数据库中该错误已累计 30+ 次,覆盖 4 个不同供应商。

## 复现步骤

1. 添加一个自定义供应商,codex runtime,线协议选 openai-chat,指向任意第三方 OpenAI 兼容网关
2. 选择该供应商下任意模型(多模态与否都一样)
3. 在对话中粘贴一张图片并发送
4. 立即返回上述 400,图片从未送达上游

## 实际行为

错误来自 Cindy 本地桥接层,而非上游网关:

- `codex.exe` (0.145.0) 中搜索 `Chat Completions bridge` / `unsupported_feature` → **0 命中**,codex 侧只有 input_image/image_url 的正常序列化结构
- `app.asar` 中命中:`UNSUPPORTED_RESPONSES_FEATURE_MESSAGE_PREFIX`,抛出点日志标记为 `responses-chat bridge rejected unsupported feature`
- 佐证:user 消息与 error 消息的 `created_at` 仅相差 1ms,请求根本没发出去

### 根因

桥接层**实际具备**完整的图片转换能力,`imagePart()` 能正确把 Responses 的 `input_image` 转成 Chat 的 `{type:"image_url",image_url:{url}}` 并支持 detail 字段。但这段代码被一个能力开关拦在门外。

`mediaPart()` 中:

```js
if(isResponsesImageContentPartType(r.type)){
if(!hasImageSource(r))return;
if(e.imageInput!=="image_url"){ if(n)throw new UnsupportedResponsesFeatureError(String(r.type)); return }
return imagePart(r)
}
```

而默认能力集里**根本没有 `imageInput` 字段**:

```js
const CHAT_BRIDGE_DEFAULT_CAPABILITIES={developerRole:"system",parallelToolCalls:!0,
maxTokensField:"max_tokens",reasoningField:"none",streamUsage:!0,
passthroughFields:[...],toolCallReasoningPlaceholder:!0,forceAutoToolChoice:!0};
```

即 `imageInput === undefined`,恒不等于 `"image_url"`,恒抛错。

它只在命中硬编码白名单时才被赋值:

```js
function chatBridgeCapabilitiesForRoute(r,e,n=CHAT_BRIDGE_DEFAULT_CAPABILITIES){
return isVerifiedImageChatRoute(r,e)?{...n,imageInput:"image_url"}:n }
```

`isVerifiedImageChatRoute` 要求 https,且**同时**匹配特定 host + 特定 model:

- `model === "kimi-k3"` 且 host ∈ {api.moonshot.cn, api.moonshot.ai}
- `isDoubaoVisionModel(m)` 且 host 匹配 `/^ark\.[a-z0-9-]+\.volces\.com$/`
- `m ∈ {qwen3.6-flash, qwen3.7-plus, qwen3.8-max-preview}` 且 host ∈ dashscope 三 host
- 其余一律 `return false`

因此任何第三方或自建的 OpenAI 兼容网关,即使下游模型完全支持视觉,也无法传图。这是白名单设计的固有结果,用户无论怎么配置都绕不过。

## 期望行为

用户配置的 openai-chat 供应商应当能够发送图片。按优先级建议:

1. **提供供应商级的 imageInput 手动开关**,把判断权交给用户 —— 用户比白名单更清楚自己的网关能力
2. **失败降级**:上游若真的拒绝 image_url,回退为文字占位提示,而不是整轮 400 让对话中断
3. 若坚持白名单,至少让报错文案明确指出「当前线路未启用图片输入,请改用 responses / anthropic-messages 协议」,而不是含糊的 `unsupported_feature`

## 补充

MCP 方案无法绕过此问题:图片需先随 user turn 送达模型才谈得上调用工具,而请求在桥接层就 400 了,MCP 没有介入时机。

实测单点等长补丁(令 `chatBridgeCapabilitiesForRoute` 无条件返回 `imageInput:"image_url"`)可解除限制,说明转换链路本身是通的,缺的只是放行条件。
---
**版本区域**: CN
**OS**: win32 x64 (10.0.22631)
**界面语言**: zh-CN

Contributor guide

Open the contributing guide

Research direction

Locate the bridge implementation in app.asar and inspect chatBridgeCapabilitiesForRoute, mediaPart(), imagePart(), and isVerifiedImageChatRoute. Reproduce the request with a custom openai-chat provider and an image, then trace whether the image reaches the upstream gateway instead of being rejected locally. Done means the chosen provider capability behavior and failure handling match the issue’s expected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
electron, typescript
Domain
api, backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.