Lightning-AI / Lightning-AI/lightning-thunder
Determine the configuration and setup end-to-end test and benchmark unmodified SGLang as baseline.
@kshitij12345 is already working on this.
Since Oct 9, 2025.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
For the models (Llama 4, GPT OSS, DeepSeek V3.1, and Qwen3-Next), we should determine the configuration of interest and write a script to test and demonstrate an end-to-end inference with default unmodified SGLang code.
NOTE:
* We are planning to integrate with sglang and sglang may have it's own implementation of this model. ([Eg](https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/models/llama4.py))
* While setting up the CI job, we should remember that the end goal is to also support nvFP4. So we need figure out how to set this up in the CI with a machine that supports nvFP4 to track any regression.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.