OpenGVLab / OpenGVLab/InternVideo
[Help requested] Inference InternVideo2_clip model.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.4k
- Forks
- 160
- PR merge metrics
- No merged PRs in 30d
Description
Hello InternVideo team,
You guys have done a great job with this project!
In your paper, you use the Stage 2 model for the task of temporal grounding on QVHighlight [Lei et al., 2021] and Charade-STA [Gao et al., 2017]. I have a question, why not use the CLIP version for this purpose?
As you mentioned in one of the issues I saw, the CLIP one is fine-tuned from Stage 2 to support more applications (with the powerful InternVL text encoder).
Am I correct in understanding that you kept the video encoder model unchanged, and the BERT-L was replaced with another text encoder? If so, where can I obtain the weights for this encoder?
In the evaluation script, you use "your_model_path/internvl/internvl_c_13b_224px.pth", there is no such model in the InternVL repository.
@Andy1621
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with InternVideo2/multi_modality/scripts/evaluation/clip/zero_shot/1B/config_charades_mc.py and compare its referenced internvl_c_13b_224px.pth path with the linked InternVL repository. Trace the CLIP and Stage 2 model descriptions to verify whether the video and text encoders differ, then document the correct encoder weights or explain that they are unavailable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100