sgl-project / sgl-project/sglang
[Bug] Models fail to produce non-empty answer while evaluated for reasoning on long context.
- Dominant language
- Python
- Stars
- 36.1k
- Forks
- 9k
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 239
Description
### Checklist
- [x] I searched related issues but found no solution.
- [x] The bug persists in the latest version.
- [x] Issues without environment info and a minimal reproducible demo are hard to resolve and may receive no feedback.
- [x] If this is not a bug report but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed.
- [x] Please use English. Otherwise, it will be closed.
### Describe the bug
While ran on SGLang, the reasoning models (e.g. GLM-5.1, Kimi-K2.5) most of the time fail to produce output both in the chain-of-thought and/or conclusive stages. That is measured with testing on the LongBench V2 questions.
Both on CUDA and ROCm platforms, the model doesn't always produce an answer, the rate of successful thought-conclusion chains on a given question is not zero though. I.e. if the same question is asked repeatedly, eventually the answer is non-empty.
If the collected non-empty responses are evaluated for correctness, the resulting level of accuracy matches SOTA.
### Reproduction
sglang serve --model-path zai-org/GLM-5.1 --tp-size 8 --host 127.0.0.1 --port {port of choice} --trust-remote-code --reasoning-parser glm45
Client:
python3 ./pred.py --model GLM-5.1 --cot --n_proc 503
### Environment
Docker:
lmsysorg/sglang:latest (as of July, 15, 2026)
Contributor guide
Research direction
Start by reproducing the issue with the provided sglang serve command and pred.py on LongBench V2, noting whether empty chain-of-thought or conclusion responses occur. Compare repeated requests and both CUDA and ROCm runs. Done means reasoning evaluations consistently produce non-empty thought and/or conclusive output without reducing the reported accuracy.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100