abetlen / abetlen/llama-cpp-python
Failed to load shared library \venv\Lib\site-packages\llama_cpp\llama.dll
- Langage dominant
- Python
- Étoiles
- 10.6k
- Forks
- 1.4k
- Métriques de merge des PR
- Métriques de PR en attente
Description
Hi,
I am running llama-cpp-python on surface book 2 having i7 with nvidea geforce gtx 1060.
I installed vc++, cuda drivers 12.4
Running on Python 3.11.3
Compiled llama using below command on MinGW bash console
```
CUDACXX="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\bin\nvcc.exe" CMAKE_ARGS="-DLLAMA_CUBLAS=on -DCMAKE_CUDA_ARCHITECTURES=all-major" FORCE_CMAKE=1
pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade --verbose
```
It ran successfully and yielded llama.dll
However, when i try to load, it is throwing error
```
File "D:\My_Tech\GenAIPlay\venv\Lib\site-packages\llama_cpp\llama_cpp.py", line 72, in _load_shared_library
raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}")
RuntimeError: Failed to load shared library 'D:\My_Tech\GenAIPlay\venv\Lib\site-packages\llama_cpp\llama.dll': Could not find module 'D:\My_Tech\GenAIPlay\venv\Lib\site-packages\llama_cpp\llama.dll' (or one of its dependencies). Try using the full path with constructor syntax.
```
I tried fixing it as per other suggestions by modifying llama_cpp.py where the error was throwing as below but didn't work
return ctypes.CDLL(str(_lib_path),winmode=0)
I did set environment variables as well, still couldn't work.
Can you please help how can i fix it ?
```
$ echo $CUDA_PATH
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4
(venv)
umapa@UMA-SB2 MINGW64 /d/My_Tech/GenAIPlay
$ echo $PATH
D:\My_Tech\GenAIPlay\venv/Scripts:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\libnvvp;C:\Python311\Scripts\;C:\Python31
1\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Users\umapa\AppData\Local\Packages\PythonSoftwareFoundatio
n.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\Scripts;C:\Program Files\Git\cmd;C:\Program Files\Docker\Docker\resources\bin;C:\Program Files\nodejs\;C:\ProgramData\chocolatey\bi
n;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files\NVIDIA Corporation\Nsight Compute 2024.1.0\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Prog
ram Files (x86)\Incredibuild;C:\Users\umapa\AppData\Local\Microsoft\WindowsApps;C:\Users\umapa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages
\Python39\Scripts;;C:\Users\umapa\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\umapa\AppData\Roaming\npm;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\lib;D:\My_Tech\GenAIPlay\venv\Lib\site-packages\llama_cpp
```
```INSTALLATION NOTES
*** Installing project into wheel...
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/ggml_shared.lib
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/ggml_shared.dll
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/cmake/Llama/LlamaConfig.cmake
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/cmake/Llama/LlamaConfigVersion.cmake
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/ggml.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/ggml-alloc.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/ggml-backend.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/ggml-cuda.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/llama.lib
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/llama.dll
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/llama.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/convert.py
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/convert-lora-to-ggml.py
-- Installing: C:/Users/umapa/AppData/Local/Temp/tmpffhzeu6k/wheel/platlib/llama_cpp/llama.lib
-- Installing: C:/Users/umapa/AppData/Local/Temp/tmpffhzeu6k/wheel/platlib/llama_cpp/llama.dll
-- Installing: C:/Users/umapa/AppData/Local/Temp/pip-install-cn0mqw5n/llama-cpp-python_bd51aa929f42429a9180b8d6bd519841/llama_cpp/llama.lib
-- Installing: C:/Users/umapa/AppData/Local/Temp/pip-install-cn0mqw5n/llama-cpp-python_bd51aa929f42429a9180b8d6bd519841/llama_cpp/llama.dll
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/llava.lib
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/llava.dll
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/llava-cli.exe
-- Installing: C:/Users/umapa/AppData/Local/Temp/tmpffhzeu6k/wheel/platlib/llama_cpp/llava.lib
-- Installing: C:/Users/umapa/AppData/Local/Temp/tmpffhzeu6k/wheel/platlib/llama_cpp/llava.dll
-- Installing: C:/Users/umapa/AppData/Local/Temp/pip-install-cn0mqw5n/llama-cpp-python_bd51aa929f42429a9180b8d6bd519841/llama_cpp/llava.lib
-- Installing: C:/Users/umapa/AppData/Local/Temp/pip-install-cn0mqw5n/llama-cpp-python_bd51aa929f42429a9180b8d6bd519841/llama_cpp/llava.dll
*** Making wheel...
*** Created llama_cpp_python-0.2.56-cp311-cp311-win_amd64.whl...
Building wheel for llama-cpp-python (pyproject.toml) ... done
Created wheel for llama-cpp-python: filename=llama_cpp_python-0.2.56-cp311-cp311-win_amd64.whl size=22345276 sha256=ffffb1a35fc1e2b8a49a1c90d0e4f4a490ed71c298a29cd468816bbb6251aad7
Stored in directory: C:\Users\umapa\AppData\Local\Temp\pip-ephem-wheel-cache-px49kwop\wheels\f5\48\62\014b1a3c38f77df21219f81ed63ca4c09531d52a205b15d8e4
Successfully built llama-cpp-python
Installing collected packages: typing-extensions, numpy, MarkupSafe, diskcache, jinja2, llama-cpp-python
```
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.