OpenGVLab / OpenGVLab/InternVideo
Clip model size is too small
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.4k
- Forks
- 160
- PR merge metrics
- No merged PRs in 30d
Description
Hello, really appreciate for your great work.
https://github.com/OpenGVLab/InternVideo/blob/main/InternVideo2/multi_modality/MODEL_ZOO.md
I checked that you guys wrote "We also learn a CLIP-style InternVideo2 indicated by InternVideo2clip. It is post-pretrained from InternVideo2s2 by only preserving video and text encoders and contrastive loss." in your paper.
But I found out that this model [InternVideo2-CLIP-1B-224p-f8] in huggingface is too small, like just a few MB. And according to the right before issue I noticed that that pth file in huggingface is "add on parameter", not a full parameter.
- So as i understood, there might be only clip model that post trained after stage2 right?
- I want to know how can I initialize that clip model and utilize. I want to get clip score from that model. It would be really grateful if you let me know exact way to do that. ( It is quite confusing no matter how much time i refer to your readme and demo.ipynb file.)
Thank you in advance.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with InternVideo2/multi_modality/MODEL_ZOO.md and the README and demo.ipynb mentioned in the issue. Trace how the InternVideo2-CLIP-1B-224p-f8 checkpoint is loaded and determine how its add-on parameters relate to the base model. Done means documenting the exact initialization and clip-scoring steps, including which files or weights are required.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100