github / github/copilot-cli

Automatic model routing: route image inputs to vision-capable model, return text to text-only model

オープン
#3,295 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
area:models
主要言語
Shell
スター
11.2k
フォーク
1.9k
平均マージ
14時間 16分
マージ済み PR(30日)
6

説明

### Describe the feature or problem you''d like to solve

When using a text-only model like DeepSeek as the primary model, image inputs are either ignored or cause errors, because DeepSeek lacks vision capabilities. Users must manually switch models with /model every time they want to include an image, which breaks workflow flow.

### Proposed solution

Automatic model routing based on input modality. When a user''s prompt includes images (pasted, drag-and-drop, or @-referenced), Copilot CLI should:

1. Detect that the primary model doesn''t support vision
2. Auto-route the image(s) to a vision-capable model (e.g., GPT-4o, Claude Sonnet 4.5) configured as the "vision fallback"
3. Receive a text description of the image(s) from the vision model
4. Inject that text description into the prompt sent to the primary (text-only) model
5. The user only sees the final response from their chosen primary model, with the image description transparently provided as context

Configuration (example .copilot/config or copilot-instructions.md):
```
vision_fallback_model: "gpt-4o"
vision_fallback_behavior: "describe_and_forward"
```

### Example prompts or workflows

1. User has DeepSeek selected. Pastes a screenshot of a UI bug and types "Fix this layout issue". CLI detects image + text-only model -> sends image to GPT-4o -> GPT-4o returns text description -> that description is prepended to DeepSeek''s prompt -> DeepSeek fixes the code.

2. User @-references a diagram.png file: "Implement this architecture diagram". Same routing flow - architecture described in text, DeepSeek implements.

3. /model deepseek is active. User pastes error screenshot. Without needing to switch models, user gets a code fix.

4. Works in reverse too - if the primary model IS vision-capable, no routing occurs; image is sent directly.

5. Configurable: user can disable auto-routing or choose which vision model to use as fallback.

### Additional context

- This is inspired by the existing sub-agent and delegation architecture already present in Copilot CLI
- Reduces friction for users who prefer text-only models for cost/performance but occasionally need vision capabilities
- Could be implemented as a lightweight pre-processing step before the main model invocation
- Related: the /fleet and custom agents infrastructure could potentially be leveraged for this

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

まず既存のサブエージェントと委譲のアーキテクチャを読み、次に issue で言及されている /fleet と custom agents のインフラを調査します。画像検出、vision 対応フォールバックの設定、説明の転送、ルーティングなしのパスをどのように連携させるべきかを定義します。画像入力が必要に応じて設定済みの vision 対応フォールバックに到達し、ユーザーにはプライマリモデルの最終レスポンスだけが返されれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
shell
領域
ai, cli
issue の種類
機能追加
難易度
5/5
見積もり時間
1週間以上
活発さ
静か
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。