DAMO-NLP-SG / DAMO-NLP-SG/VideoLLaMA3
The A100-80G graphics card is running the videollama3-7b model, and the error message is that the video memory is overflowing!!!
- Dominant language
- Jupyter Notebook
- Stars
- 1.2k
- Forks
- 89
- PR merge metrics
- No merged PRs in 30d
Description
Hi!
Can you help me solve two problems?
1. I am using the Videollama3-7b model to fine tune my own video data set based on A100 graphics card. After running the stage3.sh script, why does the video memory surge after a few epochs, exceeding 80g causing video memory overflow? Why did the first few epochs work?
2, fine tuning your own video data only need to run stage4 of the stage4.sh script?
The parameters in my stage4_2b.sh script are configured as follows
Thanks for you help!
各位老师好,我想请教两个问题:
1、我使用videollama3-7b模型基于A100显卡在我自己的视频数据集进行微调,运行stage3.sh或stage4.sh脚本后,为什么显存在跑了几个epoch后会激增,超过80g导致显存溢出?为什么前几个epoch能正常运行?
2、微调自己的视频数据只需要运行第四阶段的stage4.sh脚本么?
我的stage4_2b.sh脚本中参数配置如下:


Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the referenced stage3.sh, stage4.sh, and stage4_2b.sh scripts, including the parameters shown in the screenshots, then reproduce the training run while tracking memory across epochs. Done means identifying the cause of the delayed memory surge and documenting whether custom video fine-tuning requires stage 4 alone or additional stages.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100