DAMO-NLP-SG / DAMO-NLP-SG/VideoLLaMA2

How to Extract Video and Audio Features Using VideoLLaMA2?

Open
#155 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.3k
Forks
90
PR merge metrics
No merged PRs in 30d

Description

Hello,

I have a set of pre-chunked video clips, and I would like to use VideoLLaMA2 to extract video and audio feature vectors for each chunk. However, I couldn't find clear documentation on how to achieve this using the pre-trained model.

Could you provide guidance on:

- Extracting video features (frame embeddings) from the vision encoder used in VideoLLaMA2?
- Extracting audio features?
- The recommended process for batch processing multiple video chunks efficiently.

I would appreciate any code references or scripts within the repo that facilitate feature extraction. I appreciate any help you can provide. Thanks, in advance!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.