cloudinary / cloudinary/ssimulacra2
CID22: full dataset annotations not included in download
- Dominant language
- C++
- Stars
- 297
- Forks
- 12
- PR merge metrics
- No merged PRs in 30d
Description
The CID22 full dataset download (7.2 GB, 250 references, ~22k compressed images) only includes `CID22_validation_set.csv` with annotations for the 49-reference validation subset (4,292 scored pairs).
The remaining ~17.7k pairs from the other 201 references have no published human scores. The [CID22 labs page](https://cloudinary.com/labs/cid22) doesn't mention that the full set annotations are withheld — it describes the dataset as containing "22,000 subjective scores" but only ~4,300 are in the download.
Could the full set annotations (MCOS/RMOS/Elo scores for all 250 references) be published? This would make CID22 much more useful as a benchmark for independent metric development and comparison.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.