Bluefog-Lib / Bluefog-Lib/bluefog
Left-over data for BlueFog optimizers using num_step_per_communication
Open
investigation
- Dominant language
- Python
- Stars
- 257
- Forks
- 50
- PR merge metrics
- No merged PRs in 30d
Description
When the batch_size cannot be exactly divided by num_step_per_communication in the BlueFog optimizers, each epoch will leave some data to the next epoch for communication, which may result in a model not fully using the entire dataset in the last epoch. In addition, the mixed data usage between adjacent epochs is also not desired.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.