abetlen / abetlen/llama-cpp-python

Failed to load shared library \venv\Lib\site-packages\llama_cpp\llama.dll

Abierto
#1,280 6 comentarios 0 reacciones 0 asignados Ver en GitHub
bug
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

Hi,

I am running llama-cpp-python on surface book 2 having i7 with nvidea geforce gtx 1060.
I installed vc++, cuda drivers 12.4
Running on Python 3.11.3
Compiled llama using below command on MinGW bash console
```
CUDACXX="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\bin\nvcc.exe" CMAKE_ARGS="-DLLAMA_CUBLAS=on -DCMAKE_CUDA_ARCHITECTURES=all-major" FORCE_CMAKE=1
pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade --verbose
```

It ran successfully and yielded llama.dll
However, when i try to load, it is throwing error

```
File "D:\My_Tech\GenAIPlay\venv\Lib\site-packages\llama_cpp\llama_cpp.py", line 72, in _load_shared_library
raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}")
RuntimeError: Failed to load shared library 'D:\My_Tech\GenAIPlay\venv\Lib\site-packages\llama_cpp\llama.dll': Could not find module 'D:\My_Tech\GenAIPlay\venv\Lib\site-packages\llama_cpp\llama.dll' (or one of its dependencies). Try using the full path with constructor syntax.
```
I tried fixing it as per other suggestions by modifying llama_cpp.py where the error was throwing as below but didn't work

return ctypes.CDLL(str(_lib_path),winmode=0)

I did set environment variables as well, still couldn't work.
Can you please help how can i fix it ?
```
$ echo $CUDA_PATH
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4
(venv)
umapa@UMA-SB2 MINGW64 /d/My_Tech/GenAIPlay
$ echo $PATH
D:\My_Tech\GenAIPlay\venv/Scripts:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\libnvvp;C:\Python311\Scripts\;C:\Python31
1\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Users\umapa\AppData\Local\Packages\PythonSoftwareFoundatio
n.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\Scripts;C:\Program Files\Git\cmd;C:\Program Files\Docker\Docker\resources\bin;C:\Program Files\nodejs\;C:\ProgramData\chocolatey\bi
n;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files\NVIDIA Corporation\Nsight Compute 2024.1.0\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Prog
ram Files (x86)\Incredibuild;C:\Users\umapa\AppData\Local\Microsoft\WindowsApps;C:\Users\umapa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages
\Python39\Scripts;;C:\Users\umapa\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\umapa\AppData\Roaming\npm;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\lib;D:\My_Tech\GenAIPlay\venv\Lib\site-packages\llama_cpp
```

```INSTALLATION NOTES
*** Installing project into wheel...
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/ggml_shared.lib
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/ggml_shared.dll
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/cmake/Llama/LlamaConfig.cmake
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/cmake/Llama/LlamaConfigVersion.cmake
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/ggml.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/ggml-alloc.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/ggml-backend.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/ggml-cuda.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/llama.lib
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/llama.dll
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/llama.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/convert.py
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/convert-lora-to-ggml.py
-- Installing: C:/Users/umapa/AppData/Local/Temp/tmpffhzeu6k/wheel/platlib/llama_cpp/llama.lib
-- Installing: C:/Users/umapa/AppData/Local/Temp/tmpffhzeu6k/wheel/platlib/llama_cpp/llama.dll
-- Installing: C:/Users/umapa/AppData/Local/Temp/pip-install-cn0mqw5n/llama-cpp-python_bd51aa929f42429a9180b8d6bd519841/llama_cpp/llama.lib
-- Installing: C:/Users/umapa/AppData/Local/Temp/pip-install-cn0mqw5n/llama-cpp-python_bd51aa929f42429a9180b8d6bd519841/llama_cpp/llama.dll
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/llava.lib
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/llava.dll
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/llava-cli.exe
-- Installing: C:/Users/umapa/AppData/Local/Temp/tmpffhzeu6k/wheel/platlib/llama_cpp/llava.lib
-- Installing: C:/Users/umapa/AppData/Local/Temp/tmpffhzeu6k/wheel/platlib/llama_cpp/llava.dll
-- Installing: C:/Users/umapa/AppData/Local/Temp/pip-install-cn0mqw5n/llama-cpp-python_bd51aa929f42429a9180b8d6bd519841/llama_cpp/llava.lib
-- Installing: C:/Users/umapa/AppData/Local/Temp/pip-install-cn0mqw5n/llama-cpp-python_bd51aa929f42429a9180b8d6bd519841/llama_cpp/llava.dll
*** Making wheel...
*** Created llama_cpp_python-0.2.56-cp311-cp311-win_amd64.whl...
Building wheel for llama-cpp-python (pyproject.toml) ... done
Created wheel for llama-cpp-python: filename=llama_cpp_python-0.2.56-cp311-cp311-win_amd64.whl size=22345276 sha256=ffffb1a35fc1e2b8a49a1c90d0e4f4a490ed71c298a29cd468816bbb6251aad7
Stored in directory: C:\Users\umapa\AppData\Local\Temp\pip-ephem-wheel-cache-px49kwop\wheels\f5\48\62\014b1a3c38f77df21219f81ed63ca4c09531d52a205b15d8e4
Successfully built llama-cpp-python
Installing collected packages: typing-extensions, numpy, MarkupSafe, diskcache, jinja2, llama-cpp-python
```

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.