ChenRocks / ChenRocks/UNITER

Image-text retrieval results can't be reproduced

Open
#44 1 comment 4 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
799
Forks
111
PR merge metrics
No merged PRs in 30d

Description

Thanks for your contribution!
I run your code and can not reproduce the performance in your paper.
There my results in two different setting:
Finetuning Image-text retrieval with "train-itm-flickr-base-8gpu.json"
图片

Finetuning Image-text retrieval with"train-itm-flickr-base-16gpu-hn.json"
图片

There is a gap between the results and those in your paper. What's the difference between the experiment you do and the code you released? For example, training steps and learning rate. Thanks!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running the two cited configurations, train-itm-flickr-base-8gpu.json and train-itm-flickr-base-16gpu-hn.json, and compare their training steps, learning rate, and reported results with the paper. Check the issue's attached result images and document the experiment differences needed to make the released code reproduce the paper's performance.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.