AnswerDotAI / AnswerDotAI/byaldi
Model is not being offloaded from VRAM
- Dominant language
- Python
- Stars
- 851
- Forks
- 91
- PR merge metrics
- No merged PRs in 30d
Description
I am trying to run the model in Jupyter notebook.

1. In the above iteration I haven't initialized the model.

2. Now I run the cell the model is loaded and it is showing 6GB of vram occupied right.

3. Now when I run the cell again the vram usage is doubled.
4. In the consequent runs the model is not occupying more than 12GB but what's interesting thing I have observed is when I am running that inside a loop for suppose I want to create an Index for each file I have, I don't have any other option than do this but this is causing the model to give me vram issues. How do I remove them from vram, I tried torch cuda cache free, tried to delete the variable none isn't working for me. Can you please help or is there something I am doing wrongly ?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.