google-research / google-research/big_vision

cannot achieve the claimed zero-shot retrieval results of SigLIP2 ViT/B16-224 on COCO and Flickr30K in your paper

Open
#166 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
3.5k
Forks
227
PR merge metrics
No merged PRs in 30d

Description

On COCO and Flickr30K, the zero-shot SigLIP2 ViT/B16-224 model fails to achieve the retrieval performance claimed in the paper: in the paper, COCO (t2i-R@1: 52.1, i2t-R@1: 68.9) and Flickr30K (t2i-R@1: 80.7, i2t-R@1: 93.0). However, in my own experiments, the results are COCO (t2i-R@1: 48.58, i2t-R@1: 64.96) and Flickr30K (t2i-R@1: 73.66, i2t-R@1: 89.30).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.