lightly-ai / lightly-ai/lightly
Reproducing SimCLR on ImageNet-1K from provided pretrained ckpt
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.8k
- Forks
- 367
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 5
Description
I download the ckpt provided in this project which is named "epoch=99-step=500400.ckpt" and use the linear_eval code in benchmark script to evaluate it. However, with the default settings (lr=0.1) I got NaN when training. I changed it into 1e-4 to avoid NaN but the accuracy is still very low, only about 1%, very far from the reported accuracy. I also tried the KNN_eval and the accuracy is also only about 0.5%.
How can I solve this problem? Is there any other change required in this code? I have made sure that the checkpoint is loaded correctly.
Thank you so much
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the benchmark script's linear_eval and KNN_eval paths, then compare their default settings with the provided epoch=99-step=500400.ckpt and the reported ImageNet-1K setup. Reproduce the NaN at lr=0.1 and the low accuracies at lr=1e-4 and in KNN_eval; done means identifying the required configuration or code change that makes evaluation reproduce the reported results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100