ChenRocks / ChenRocks/UNITER

What is the gather index?

Open
#86 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
799
Forks
111
PR merge metrics
No merged PRs in 30d

Description

**This is more of a question than issue.**

In model.py at line number 330 gather index has been used to re-orient text and image embedding after concatenation. I am trying to understand what is this gather index conceptually. I am trying to find the same in the paper. From the paper it seems image and text features are concatenated. So it will be helpful to know what these gather index vectors represents and how to create them for a custom dataset.

Thanks.

Contributor guide

No contributing guide indexed for this repository

Research direction

Read model.py around line 330 and compare the concatenation and re-orientation logic with the paper. Trace how the gather index vectors are formed in the existing data path. Done means documenting what they represent and how they should be created for a custom dataset.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.