The image is cut from a request to a custom model
- Ngôn ngữ chính
- Rust
- Star
- 54.2k
- Fork
- 6.2k
- Merge trung bình
- 3 ngày 4 giờ
- Pull request đã merge (30 ngày)
- 240
Mô tả
**Describe the bug**
Continue https://github.com/aaif-goose/goose/issues/10311 and https://github.com/aaif-goose/goose/pull/11496
Look to the screenshot. Sometime i got `[image omitted: model does not support vision]`
---
**To Reproduce**
Steps to reproduce the behavior:
0. Install 1.48.0
1. Go to 'New chat'
2. Drag and drop image
3. Chat `what is it?`
4. See thinking block
---
**Expected behavior**
The model returns the contents of the image
---
**Screenshots**
---
**Please provide the following information**
- **OS & Arch:** Windows 10
- **Interface:** UI
- **Version:** 1.48.0
- **Extensions enabled:** Computer Controller
- **Provider & Model:** Custom vLLM server with multimodal llm qwen3.8-27b
---
**Additional context**
Full provider config
```json
{
"name": "custom_qwen3_8-27b",
"engine": "openai",
"display_name": "qwen3.8-27b",
"description": "Custom qwen3.8-27b provider",
"api_key_env": "",
"base_url": "http://*:8000",
"models": [
{
"name": "qwen3.8-27b",
"context_limit": 524288, // change from default
"input_token_cost": null,
"output_token_cost": null,
"currency": null,
"supports_cache_control": null,
"reasoning": false,
"supports_vision": true // add to default
}
],
"headers": null,
"timeout_seconds": null,
"supports_streaming": true,
"requires_auth": false,
"catalog_provider_id": null,
"base_path": null,
"env_vars": null,
"dynamic_models": null,
"skip_canonical_filtering": true, // change from default
"model_doc_link": null,
"setup_steps": [],
"fast_model": null,
"preserves_thinking": true,
"emit_clear_thinking": false,
"setup": null
}
```
curl request is just working
```
{
"model": "qwen3.8-27b",
"messages": [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"
}
},
{"type": "text", "text": "what is it?"}
]
}
]
}
```
and return correct answer
```json
{
"role": "assistant",
"content": "\n\nIt's a photo of a woman and her dog on a sandy beach at sunset (or sunrise), with the warm, low sun flaring in from the right.\n\nThe main moment in the picture is the interaction between them:\n\n- **The dog** is a yellow ...",
"refusal": null,
"annotations": null,
"audio": null,
"function_call": null,
"reasoning": "The user's query is just \"what is it?\" which is ambiguous. They want me to identify the image content. Let me look carefully.\n\nThe ..."
}
```
Hướng dẫn đóng góp
Hướng nghiên cứu
The issue is about vision support for a custom model. Look at the provider configuration and the linked PR #11496 to understand how vision support is determined. Check the code that handles image attachments and sends requests to the OpenAI-compatible endpoint. The bug likely involves the model's `supports_vision` flag or how the image data is formatted in the request. Start by examining the provider engine logic and the request builder for multimodal messages.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- rust
- Lĩnh vực
- ai, backend
- Loại issue
- Lỗi
- Độ khó
- 3/5
- Thời gian dự kiến
- 1-2 ngày
- Mức độ hoạt động
- Sôi nổi
- Độ rõ ràng
- Khá rõ ràng
- Mức phù hợp với người mới
- 50/100