facebookresearch / facebookresearch/moco

Does MOCO collapses under simpler augmentation?

Open
#79 4 comments 2 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
5.1k
Forks
802
PR merge metrics
No merged PRs in 30d

Description

Recently I have been applying your implementation to simpler datasets like CIFAR10. A strong augmentation will hurt the moco's convergence on a smaller dataset, so I simplified the augmentation when training MOCO on CIFAR10:
transforms.RandomCrop(32, padding=4),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
normalize.

Under this simpler setting, the training top1 and top5 goes up to nearly 100% at about 20th epoch. The other hyperparameters are almost unchanged, except: batch size=512, lr=0.015, arch=resnet-18, tau=0.1, k=4096.

Is this a phenomenon of collapsing?

Thanks.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.