Change LayerwiseDistributedOptimizer's Allgather_v into Allgather
Open
enhancement
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 4.5k
- Avg merge
- 4d 6h
- Merged PRs (30d)
- 271
Description
Currently LayerwiseDistributedOptimizer is using AG_v, we can change it to AG with paddings. Implement and analyze this change
Contributor guide
Assessment
This issue has not been assessed yet.