DAMO-NLP-SG / DAMO-NLP-SG/VideoLLaMA3

which stage to train for videos?

Open
#84 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
1.2k
Forks
89
PR merge metrics
No merged PRs in 30d

Description

I only have videos with description and I want to ado video understanding. can I directly start fine tuning from stage 3? if yes where will we get the weights till stage 2? with this particular command, no weights are downloaded..only some files are downloaded.
python scripts/convert_hf_checkpoint.py --model_path DAMO-NLP-SG/VideoLLaMA3-7B --save_path weights/videollama3_7b_local

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading scripts/convert_hf_checkpoint.py and checking the DAMO-NLP-SG/VideoLLaMA3-7B checkpoint referenced in the command. Determine which files the conversion downloads and how they relate to the stage 2 and stage 3 training prerequisites. Done means the required weights and the correct training stage are documented or the missing-weight behavior is reproducible.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.