facebookresearch / facebookresearch/moco
Does MOCO collapses under simpler augmentation?
Open
- Dominant language
- No language data
- Stars
- 5.1k
- Forks
- 802
- PR merge metrics
- No merged PRs in 30d
Description
Recently I have been applying your implementation to simpler datasets like CIFAR10. A strong augmentation will hurt the moco's convergence on a smaller dataset, so I simplified the augmentation when training MOCO on CIFAR10:
transforms.RandomCrop(32, padding=4),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
normalize.
Under this simpler setting, the training top1 and top5 goes up to nearly 100% at about 20th epoch. The other hyperparameters are almost unchanged, except: batch size=512, lr=0.015, arch=resnet-18, tau=0.1, k=4096.
Is this a phenomenon of collapsing?
Thanks.
Contributor guide
Assessment
This issue has not been assessed yet.