Lightning-AI / Lightning-AI/lightning-thunder
CPU offloading tracker
Open
@beverlylytle is already working on this.
Since Aug 26, 2025.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
This issue is to track progress made toward increasing the maximum sequence length supported in training by using CPU offloading to reduce the peak GPU memory usage. Sub-tasks falling under this tracking issue:
- adding a transformer to insert off-/re-loading symbols in the forward and backward traces
- adding support for CUDA streams to allow data transfer/compute overlap
- adding profiling to verify increase in max sequence length for fixed model (set)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.