agentscope-ai / agentscope-ai/QwenPaw
[Feature]: support separate vision model routing for image inputs
- Langage dominant
- Python
- Étoiles
- 34.9k
- Forks
- 3.1k
- Merge moyen
- 1 j 15 h
- PR mergées (30 j)
- 225
Description
## Background
In QwenPaw Desktop, if the current chat model does not support multimodal/image input, an uploaded image cannot be analyzed by the Agent. The user currently has to manually switch the whole conversation to a vision-capable model, then upload the image again.
During testing, a non-vision model could not inspect an uploaded screenshot, but after switching the chat model to `gpt-5.5`, the same kind of screenshot could be recognized successfully. This suggests that the image upload/transport path works, but QwenPaw currently lacks automatic routing from a text-only chat model to a configured vision model.
## Requested capability
Please consider supporting separate model configuration for different input/task types, especially:
- Default chat model
- Vision/image model
- Optional future extensions:
- Audio model
- OCR model
- Tool reasoning model
- Embedding/memory model
For image inputs, the expected behavior could be:
1. If the current chat model supports vision, use the current model directly.
2. If the current chat model does not support vision, automatically call a configured `vision_model`.
3. The vision model returns a description/OCR/structured extraction result.
4. The main chat model then uses that result to continue the conversation.
## Why this is useful
- Users can keep a cheaper/faster/stabler model as the daily chat model.
- Expensive or specialized vision models are only used when image inputs appear.
- Users do not need to manually switch models back and forth.
- It enables cleaner multi-model workflows inside one Agent.
- It reduces confusion when an uploaded image silently fails or cannot be inspected by the current model.
## Possible configuration idea
Something like this in model/agent configuration:
```yaml
models:
default_model: xxx
vision_model: gpt-4o-or-other-vision-model
audio_model: xxx # optional
tool_reasoning_model: xxx # optional
```
Or in the Desktop UI:
- Default chat model
- Image / vision model
- Audio model (optional)
- Advanced routing rules (optional)
## Example scenario
1. User chats with a text-only model.
2. User uploads a screenshot and asks: "Can you see this?"
3. The Agent detects that the current model cannot process images.
4. QwenPaw automatically sends the image to the configured vision model.
5. The vision result is passed back to the text model.
6. The Agent answers normally without requiring manual model switching.
Thanks for considering this. It would make QwenPaw's multi-model experience much smoother.
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.