facebookresearch / facebookresearch/fairscale
[checkpoint]: error when convert sync bn is done after checkpoint
Open
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 293
- PR merge metrics
- No merged PRs in 30d
Description
@prigoyal found a bug when checkpoint is done before sync BN conversion, it fails with:
```
torch.nn.modules.module.ModuleAttributeError: 'SyncBatchNorm' object has no attribute '_checkpoint_fwd_counter'
```
reversing the order avoids this problem. It would be nice to detect this and tell users what's going on and how to fix it.
Contributor guide
Assessment
This issue has not been assessed yet.