sgl-project / sgl-project/SpecForge
Why is the TPS of eagle3-qwen in the sglang inference of single-card H20 not as high as that of the original QWEN3 when the decoding algorithm is added
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 347
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 41
Description
Hello, I'm testing the speed of 100 tokens on a single H20. The original qwen3 has 200TPS during sglang inference, while the draft model eagle3 only has 130TPS. What's the reason for this
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported 100-token comparison on a single H20: original Qwen3 at about 200 TPS versus eagle3-qwen at about 130 TPS during SGLang inference. Compare the two inference paths and decoding configuration, then document the measured cause of the throughput difference and a verified conclusion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100