huggingface / huggingface/accelerate
Plan to support FSDP2?
Open
enhancement
feature request
- Dominant language
- Python
- Stars
- 9.9k
- Forks
- 1.5k
- Avg merge
- 5d 2h
- Merged PRs (30d)
- 27
Description
FSDP2 provides smaller memory footprint, compatibility with torch compile, and more flexibility due to per param sharding. Does huggingface have plan to support FSDP2?
https://github.com/pytorch/torchtitan/blob/main/docs/fsdp.md
Contributor guide
Assessment
This issue has not been assessed yet.