sgl-project / sgl-project/sglang

[Bug] Models fail to produce non-empty answer while evaluated for reasoning on long context.

Open
#36,734 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
36.1k
Forks
9k
Avg merge
1d 5h
Merged PRs (30d)
239

Description

### Checklist

- [x] I searched related issues but found no solution.
- [x] The bug persists in the latest version.
- [x] Issues without environment info and a minimal reproducible demo are hard to resolve and may receive no feedback.
- [x] If this is not a bug report but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed.
- [x] Please use English. Otherwise, it will be closed.

### Describe the bug

While ran on SGLang, the reasoning models (e.g. GLM-5.1, Kimi-K2.5) most of the time fail to produce output both in the chain-of-thought and/or conclusive stages. That is measured with testing on the LongBench V2 questions.

Both on CUDA and ROCm platforms, the model doesn't always produce an answer, the rate of successful thought-conclusion chains on a given question is not zero though. I.e. if the same question is asked repeatedly, eventually the answer is non-empty.

If the collected non-empty responses are evaluated for correctness, the resulting level of accuracy matches SOTA.

### Reproduction

sglang serve --model-path zai-org/GLM-5.1 --tp-size 8 --host 127.0.0.1 --port {port of choice} --trust-remote-code --reasoning-parser glm45

Client:
python3 ./pred.py --model GLM-5.1 --cot --n_proc 503

### Environment

Docker:
lmsysorg/sglang:latest (as of July, 15, 2026)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with the provided sglang serve command and pred.py on LongBench V2, noting whether empty chain-of-thought or conclusion responses occur. Compare repeated requests and both CUDA and ROCm runs. Done means reasoning evaluations consistently produce non-empty thought and/or conclusive output without reducing the reported accuracy.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.