Support for PyTorch's native FSDP
Aberta
feature request
- Linguagem predominante
- Python
- Estrelas
- 572
- Forks
- 55
- Métricas de merge de PRs
- Nenhum PR com merge em 30d
Descrição
### 🚀 The feature, motivation and pitch
PyTorch's native FSDP (adapted from FairScale) is [now in beta](https://pytorch.org/blog/pytorch-1.12-released/#beta-fully-sharded-data-parallel-fsdp-api) as of v1.12. We should support it in our `TorchTrainStep`.
### Alternatives
Don't support it, since we already have a FairScale integration with FSDP.
### Additional context
[Getting Started with Fully Sharded Data Parallel](https://pytorch.org/tutorials/intermediate/FSDP_tutorial.html)
Guia de contribuição
Avaliação
Esta issue ainda não foi avaliada.