Lightning-AI / Lightning-AI/lightning-thunder
Investigate `torch.Tensor.__setitem__` as unsupported in `thunderfx`
@shino16 is already working on this.
Since Dec 9, 2025.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
Setting this operator in https://github.com/Lightning-AI/lightning-thunder/blob/f6f0496cc5f0c9071b62ef79a72ffed1a1bcb296/thunder/dynamo/utils.py#L48 gives 10x decode throughput improvement on `openai/gpt-oss-20b` and `Qwen/Qwen3-32B` inside SGLang `bench_one_batch.py`.
Repro (this only works on the NV internal [thunder-sglang-integration](https://gitlab-master.nvidia.com/kaiximatteoc/sglang/-/tree/thunder-dynamo-recompilation) codebase):
```
SGLANG_USE_THUNDER_GRAPH_RUNNER=1 python3 -m sglang.bench_one_batch --model-path openai/gpt-oss-20b --trust-remote-code --model-impl transformers --dtype bfloat16 --json-model-override-args '{"quantization_config": null}' --cuda-graph-bs 1 --tp-size 4 --tp-strategy dtensor --load-format-dummy
```
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.