[QUESTION]Does megatron-fsdp support torch.compile and cudagraphs?
Open
community-request
question
waiting-on-customer
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 4.5k
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 272
Description
Hi,
I'm interested in using megatron-fsdp for my project and would like to know about its compatibility with PyTorch optimization features:
Questions:
Does megatron-fsdp currently support torch.compile?
Does it support cudagraphs?
If not currently supported, are there any plans to add support for these features in the future?
Any information or guidance would be greatly appreciated. Thank you!
Contributor guide
Assessment
This issue has not been assessed yet.