microsoft / microsoft/XPretrain
About activitynet captions dataset in CLIP-ViP
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 511
- Forks
- 35
- PR merge metrics
- No merged PRs in 30d
Description
hello, thank you for sharing your excellent work!
I have reproduced result in msrvtt and even acquire a higher result than that in paper.
But when I tried to reproduce on activitynet captions, I found that in actnet_retrieval_vip_base_32.jsonthe vision format setting is frame instead of video, and I tried to reproduce on vision format video with sampling 32 frames setting it almost reach to r@1=20 finally.
Then I use opencv library to extract frames but it still can’t reach the result in paper.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with actnet_retrieval_vip_base_32.json and inspect how the ActivityNet Captions run selects the vision format and samples frames. Reproduce the reported video-format run with 32 frames, then compare it with the OpenCV frame extraction attempt and the result stated in the paper. Done means identifying and documenting the source of the reproduction discrepancy.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- opencv, python
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100