DAMO-NLP-SG / DAMO-NLP-SG/VideoLLaMA3
CUDA OUT OF MEMORY on Example inference code
Open
- Dominant language
- Jupyter Notebook
- Stars
- 1.2k
- Forks
- 89
- PR merge metrics
- No merged PRs in 30d
Description
First of all, thanks for the great work!
I'd like to test the model without the flash attention, using sdpa instead.
When I run the inference code in README, it makes OOM error.
Though I changed the max_frames on conversation content 180 to 16, the model still tries to allocate huge memory such as 92.4GiB.
Does anyone know how to solve it?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.