OpenGVLab / OpenGVLab/InternVideo

Similarity Scores coming very low between the video and the text features.

Open
#255 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.4k
Forks
160
PR merge metrics
No merged PRs in 30d

Description

Hi @leexinhao,
I am trying the text to video retrieval on my dataset using this https://github.com/OpenGVLab/InternVideo/blob/main/InternVideo2/multi_modality/demo_video_text_retrieval.ipynb, but the similarity scores are coming in very low between the text_features and the video_features. I am using this weight file InternVideo2-stage2_1b-224p-f4.pt and the cosine similarity I am computing by taking the dot product between the text_features and the video_features(text_features @ video_features.T).
array([0.08640765, 0.08618326, 0.08596011, 0.08578135, 0.08574679, 0.08564241, 0.08557957, 0.08552065, 0.08551717, 0.08548111], dtype=float32)

Thanks.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with InternVideo2/multi_modality/demo_video_text_retrieval.ipynb and reproduce the text-to-video retrieval flow using InternVideo2-stage2_1b-224p-f4.pt. Inspect how text_features and video_features are produced and compared, then establish whether the reported scores are expected for this dataset. Done means documenting the cause of the low scores or identifying a reproducible issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python
Domain
computer-vision, machine-learning, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.