vllm-project / vllm-project/aibrix

Make sure reasoning outputs is working as expected in aibrix

Open
#1,896 0 comments 0 reactions 0 assignees View on GitHub
kind/documentation
Dominant language
Go
Stars
5.1k
Forks
697
Avg merge
1d 19h
Merged PRs (30d)
104

Description

### 🚀 Feature Description and Motivation

We need to make sure chat level vs model startup level configuration are compatible.

```
{"enable_thinking": false}
```

1. whether aibrix configuration (gateway) has any limitation?
2. make sure both sglang & vllm are working.
3. update aibrix docs on the version/models etc

- https://docs.vllm.ai/en/latest/features/reasoning_outputs/#disabling-thinking-mode-by-default
- https://qwen.readthedocs.io/en/latest/deployment/sglang.html
- https://developer.volcengine.com/articles/7502283241375760403

### Use Case

enable reasoning mode for end users

### Proposed Solution

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with AIBrix gateway configuration and the vLLM reasoning-outputs and Qwen SGLang deployment references linked in the issue. Check chat-level and model-startup-level enable_thinking behavior for both SGLang and vLLM, then update AIBrix documentation with supported versions, models, and verified configuration.

Written by the indexing model from the issue text.

Assessment

Domain
ai-infra-agents, backend-api-design, documentation
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.