facebookresearch / facebookresearch/perception_models
Reproduce the video zero-shot evaluation results
- Dominant language
- Jupyter Notebook
- Stars
- 2.4k
- Forks
- 162
- PR merge metrics
- No merged PRs in 30d
Description
Hi, thanks for sharing the great work.
I noticed that the paper mentions:
> extending the CLIPBench zero-shot evaluation to include video datasets such as MSR-VTT and Kinetics.
While reproducing the video zero-shot evaluation results from the paper, I found that the source of these evaluation data is unclear, and benchmark test results from different sources cannot be aligned. Could you please provide download links for the video evaluation data (e.g., from the original website or Hugging Face) and details on the specific val/test splits to help reproduce the video evaluation results reported in the paper?
> from
> https://github.com/facebookresearch/perception_models/blob/3e352cca660658d4b5c90f42a7808b11469e4c66/apps/pe/clip_benchmark/datasets/builder.py#L19
> https://github.com/facebookresearch/perception_models/blob/3e352cca660658d4b5c90f42a7808b11469e4c66/apps/pe/clip_benchmark/datasets/builder.py#L27
> https://github.com/facebookresearch/perception_models/blob/3e352cca660658d4b5c90f42a7808b11469e4c66/apps/pe/clip_benchmark/datasets/builder.py#L23
> https://github.com/facebookresearch/perception_models/blob/3e352cca660658d4b5c90f42a7808b11469e4c66/apps/pe/clip_benchmark/datasets/builder.py#L27
> https://github.com/facebookresearch/perception_models/blob/3e352cca660658d4b5c90f42a7808b11469e4c66/apps/pe/clip_benchmark/datasets/builder.py#L34
Thanks!
Contributor guide
Assessment
This issue has not been assessed yet.