AnswerDotAI / AnswerDotAI/RAGatouille
Couldn't use multiple GPU to search
- Dominant language
- Python
- Stars
- 4k
- Forks
- 276
- PR merge metrics
- No merged PRs in 30d
Description
Update: I found I can use multiple GPUs to index but couldn't use multiple GPUs to search. Is this a known limitation?
I got CUDA OOM error when I index too many documents and when chunk size is too large. Are there ways to limit GPU memory usage? Thank you!
When I have 4 GPUs RAGatouille seems to only use 1. I set n_gpu=4 and it still only uses 1.
Contributor guide
No contributing guide indexed for this repository
Research direction
No file or test is named. Start by reproducing the report with n_gpu=4, comparing indexing and search behavior, and checking the CUDA out-of-memory case with large document counts or chunk sizes. Done means documenting whether multi-GPU search is supported and identifying the relevant GPU memory limitation or required change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100