LAION-AI / LAION-AI/CLIP_benchmark
Mismatch between MS-COCO val2014 images and annotations
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 814
- Forks
- 103
- PR merge metrics
- No merged PRs in 30d
Description
I am currently performing retrieval task on MS-COCO dataset, and I am using the default val2014 zip file defined in the repo, which has 40504 images in total. However, when the dataset is created, the size of it is 5k points. I believe this is due to the annotations file, which might currently point to the 2017 released version, given that the validation split is 5k. Also, val2014 is downloaded by default, even if the split points to test split. Could you please confirm if those claims are valid?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing dataset creation for the default MS-COCO val2014 archive, then inspect how the annotations file and split selection determine the dataset size. Compare the selected image and annotation releases, including test-split behavior; done means the configured assets and split produce the intended records without downloading unrelated data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100