abetlen / abetlen/llama-cpp-python
0.18 (and others?) no longer use GPU at all
Open
bug
build
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 1.4k
- PR merge metrics
- PR metrics pending
Description
https://github.com/jllllll/llama-cpp-python-cuBLAS-wheels/issues/21#issuecomment-1809000662
If I only install GPU version, fails needing llama cpp python. With both installed, never uses GPU, no mention of offlload as usual, etc.
Contributor guide
Assessment
This issue has not been assessed yet.