DAMO-NLP-SG/VideoLLaMA2

View on GitHub

VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs

Stars: 1.3k
Forks: 90
Open beginner issues: 0
Indexed issues: 87

Dominant language: Python
License: Apache-2.0
Last GitHub push: Jan 23, 2025
Latest indexed: Sep 15, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: good first issue
PR merge metrics: No merged PRs in 30d

Open All

87 open issues indexed

Multi-round chat in CLI mode Open

DAMO-NLP-SG/VideoLLaMA2#14 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
CUDA device issue. Open

DAMO-NLP-SG/VideoLLaMA2#18 · 3 comments · 2 reactions · 0 assignees · Indexed Sep 15, 2026
DeepSpeed ZeRO 3 issue Open

DAMO-NLP-SG/VideoLLaMA2#21 · 3 comments · 1 reaction · 0 assignees · Indexed Sep 15, 2026
About the training efficiency Open

DAMO-NLP-SG/VideoLLaMA2#22 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Cuda out of memory issue for simple inference Open

DAMO-NLP-SG/VideoLLaMA2#24 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
When will you realease the audio branch? Open

DAMO-NLP-SG/VideoLLaMA2#25 · 5 comments · 7 reactions · 0 assignees · Indexed Sep 15, 2026
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0 Open

DAMO-NLP-SG/VideoLLaMA2#26 · 2 comments · 1 reaction · 0 assignees · Indexed Sep 15, 2026
Could you please provide a google drive link for training dataset? Thanks a lot Open

DAMO-NLP-SG/VideoLLaMA2#29 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Inquiry VideoLLaMA2-7B Model Performance Discrepancies Open

DAMO-NLP-SG/VideoLLaMA2#34 · 10 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
'vicuna' as model_base Open

DAMO-NLP-SG/VideoLLaMA2#38 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Transcription as Input Open

DAMO-NLP-SG/VideoLLaMA2#42 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
How do I set up GPU selection in a custom dataset fine-tuning script? Open

DAMO-NLP-SG/VideoLLaMA2#43 · 3 comments · 1 reaction · 0 assignees · Indexed Sep 15, 2026
Inference print logs with warning or errors? Open

DAMO-NLP-SG/VideoLLaMA2#44 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Can the kernel version requirements be lowered from 5.5.0 to 5.4.0? Open

DAMO-NLP-SG/VideoLLaMA2#45 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Did it take almost half an hour to load the model inference weights? Is it my problem? Open

DAMO-NLP-SG/VideoLLaMA2#46 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Could you publish the loss curves for both pretraining and finetuning? Open

DAMO-NLP-SG/VideoLLaMA2#49 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Unable to install Open

DAMO-NLP-SG/VideoLLaMA2#52 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Webvid-10M (40% sampling) Open

DAMO-NLP-SG/VideoLLaMA2#56 · 2 comments · 1 reaction · 0 assignees · Indexed Sep 15, 2026
Fine tuning using "finetune_lora.sh" file Open

DAMO-NLP-SG/VideoLLaMA2#57 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
how to finetune Videollama2 chat models using QLoRA and LoRA. Open

DAMO-NLP-SG/VideoLLaMA2#58 · 9 comments · 1 reaction · 0 assignees · Indexed Sep 15, 2026
Recommend some configurations Open

DAMO-NLP-SG/VideoLLaMA2#60 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Long video error. Open

DAMO-NLP-SG/VideoLLaMA2#64 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Segmentation fault with the provided inference code Open

DAMO-NLP-SG/VideoLLaMA2#66 · 7 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Error while loading custom finetuned QLoRA model in 4 bit : size mismatch for model.mm_projector.readout.0.weight: copying a param with shape torch.Size([4096, 4096]) from checkpoint, the shape in current model is torch.Size([8388608, 1]). Open

DAMO-NLP-SG/VideoLLaMA2#71 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Problem about processor in load_pretrained_model Open

DAMO-NLP-SG/VideoLLaMA2#74 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Missing comparsion Open

DAMO-NLP-SG/VideoLLaMA2#75 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Weird "Invalid base64-encoded" Error Open

DAMO-NLP-SG/VideoLLaMA2#76 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Error while loading Mixtral based SFT MoE model VideoLLaMA2-8x7B: SafetensorError: Error while deserializing header: InvalidHeaderDeserialization Open

DAMO-NLP-SG/VideoLLaMA2#77 · 5 comments · 3 reactions · 0 assignees · Indexed Sep 15, 2026
Unable to load *ANY BASE MODEL* in 4bit Open

DAMO-NLP-SG/VideoLLaMA2#78 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Cannot reproduce results on vllava datasets Open

good first issue

DAMO-NLP-SG/VideoLLaMA2#81 · 27 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Deployment on huggingface endpoints Open

DAMO-NLP-SG/VideoLLaMA2#86 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Could you please advise when the checkpoint for the audio branch will be made public? Open

DAMO-NLP-SG/VideoLLaMA2#87 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
After fine-tuning, the model outputs repetitive phrases Open

DAMO-NLP-SG/VideoLLaMA2#89 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
ValueError: The following `model_kwargs` are not used by the model: ['images_or_videos', 'modal_list'] Open

DAMO-NLP-SG/VideoLLaMA2#90 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Videochatgpt_gen link for Test_Human_Annotated_Captions is not valid Open

DAMO-NLP-SG/VideoLLaMA2#91 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Can I run VideoLLaMA 1 in this repo? Open

DAMO-NLP-SG/VideoLLaMA2#94 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
code for batch inference Open

DAMO-NLP-SG/VideoLLaMA2#97 · 1 comment · 2 reactions · 0 assignees · Indexed Sep 15, 2026
Request for Inference Code on Custom Datasets Open

DAMO-NLP-SG/VideoLLaMA2#101 · 10 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Forward pass of the model - how to pass videos? Open

DAMO-NLP-SG/VideoLLaMA2#105 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
QLoRA fin-tunes a custom model with 4-bits, and inference the video, then we got : Open

DAMO-NLP-SG/VideoLLaMA2#106 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Can't hear the audio Open

DAMO-NLP-SG/VideoLLaMA2#112 · 14 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
How to load model model that was finetuned using qlora or lora? Open

DAMO-NLP-SG/VideoLLaMA2#113 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Inference code does not work for videos Open

DAMO-NLP-SG/VideoLLaMA2#114 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
🚀 [Release Notes] 2024.10 Open

enhancement good first issue

DAMO-NLP-SG/VideoLLaMA2#116 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
No module named 'transformers' Open

DAMO-NLP-SG/VideoLLaMA2#117 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Error When Running Multiple-model Version Demo Open

DAMO-NLP-SG/VideoLLaMA2#120 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
output strangeness Open

DAMO-NLP-SG/VideoLLaMA2#123 · 6 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Finetune model inference error Open

DAMO-NLP-SG/VideoLLaMA2#124 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Can't download the pretrained audio encoder Open

DAMO-NLP-SG/VideoLLaMA2#129 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Quick question on the need for `KeywordsStoppingCriteria` Open

DAMO-NLP-SG/VideoLLaMA2#130 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026

DAMO-NLP-SG/VideoLLaMA2

Get new issues in your inbox