DAMO-NLP-SG / DAMO-NLP-SG/VideoLLaMA3

Get Video Embedding for Video Search

Open
#44 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
1.2k
Forks
89
PR merge metrics
No merged PRs in 30d

Description

Hi, I am wondering how I can extract the video embedding instead of the video QA result from the model. I want to using video embedding for video search, after getting the video embedding result, how to project text embedding and video embedding into same latent space since visual and text and not using the same encoder?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the model's video encoder and the code path that returns video QA results; the issue does not name files, tests, or entry points. Determine whether the repository exposes video embeddings and how text and video representations are produced. Done would require a documented, tested way to obtain compatible embeddings for video search.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.