vllm-project / vllm-project/aibrix
Make sure reasoning outputs is working as expected in aibrix
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 697
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 104
Description
### 🚀 Feature Description and Motivation
We need to make sure chat level vs model startup level configuration are compatible.
```
{"enable_thinking": false}
```
1. whether aibrix configuration (gateway) has any limitation?
2. make sure both sglang & vllm are working.
3. update aibrix docs on the version/models etc
- https://docs.vllm.ai/en/latest/features/reasoning_outputs/#disabling-thinking-mode-by-default
- https://qwen.readthedocs.io/en/latest/deployment/sglang.html
- https://developer.volcengine.com/articles/7502283241375760403
### Use Case
enable reasoning mode for end users
### Proposed Solution
_No response_
Contributor guide
Research direction
Start with AIBrix gateway configuration and the vLLM reasoning-outputs and Qwen SGLang deployment references linked in the issue. Check chat-level and model-startup-level enable_thinking behavior for both SGLang and vLLM, then update AIBrix documentation with supported versions, models, and verified configuration.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai-infra-agents, backend-api-design, documentation
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100