Implement FSDP for training large datasets

未关闭

#218 创建于 2023年9月4日 1 条评论 0 个 reaction 已指派 0 人在 GitHub 查看

feat/enhancement

描述

Definition of done:
Implement training large models using FSDP to accelerate training on large datasets.

Reference: https://pytorch.org/blog/introducing-pytorch-fully-sharded-data-parallel-api/

这个 Issue 还没有评估数据。