Support for PyTorch's native FSDP
Offen
feature request
- Vorherrschende Sprache
- Python
- Sterne
- 572
- Forks
- 55
- PR-Merge-Kennzahlen
- Keine gemergten PRs in 30 T.
Beschreibung
### 🚀 The feature, motivation and pitch
PyTorch's native FSDP (adapted from FairScale) is [now in beta](https://pytorch.org/blog/pytorch-1.12-released/#beta-fully-sharded-data-parallel-fsdp-api) as of v1.12. We should support it in our `TorchTrainStep`.
### Alternatives
Don't support it, since we already have a FairScale integration with FSDP.
### Additional context
[Getting Started with Fully Sharded Data Parallel](https://pytorch.org/tutorials/intermediate/FSDP_tutorial.html)
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.