Lightning-AI / Lightning-AI/lightning-thunder
ThunderFX is slower than torch.compile for vicuna-33b-v1.3, Platypus-30B and falcon-40b with FSDP & zero3.
@IvanYashchuk is already working on this.
Since Nov 4, 2024.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 121
- PR merge metrics
- No merged PRs in 30d
Description
## 🐛 Bug
### To Reproduce
Steps to reproduce the behavior:
Please use:
2 node(s), each with 8 GPUs.
Image "INTERNAL_IMAGE:20241025"
Training script:
```
python /opt/pytorch/lightning-thunder/thunder/benchmarks/benchmark_litgpt.py \
--model_name vicuna-33b-v1.3 \
--distributed_mode fsdp \
--shard_mode zero3 \
--compile dynamo_thunder \
--checkpoint_activations False \
--low_precision_mode none \
--micro_batch_size 2
```
### Environment
system.device_product_name DGXH100
system.gpu_driver_version 535.129.03
libraries.cuda 12.6.2.004
libraries.pip.lightning 2.4.0.dev20240728
libraries.pip.lightning-thunder 0.2.0.dev0
libraries.pip.lightning-utilities 0.11.8
libraries.pip.litgpt 0.4.11
libraries.pip.nvfuser 0.2.20+git85c22a2
libraries.pip.pytorch-lightning 2.4.0
libraries.pip.torch 2.6.0a0+git96b30dc
libraries.pip.torchmetrics 1.5.1
libraries.pip.torchvision 0.19.0a0+d23a6e1
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.