abetlen / abetlen/llama-cpp-python

Failed to load shared library \venv\Lib\site-packages\llama_cpp\llama.dll

Đang mở
#1,280 6 bình luận 0 reaction 0 người được giao Xem trên GitHub
bug
Ngôn ngữ chính
Python
Star
10.6k
Fork
1.4k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

Hi,

I am running llama-cpp-python on surface book 2 having i7 with nvidea geforce gtx 1060.
I installed vc++, cuda drivers 12.4
Running on Python 3.11.3
Compiled llama using below command on MinGW bash console
```
CUDACXX="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\bin\nvcc.exe" CMAKE_ARGS="-DLLAMA_CUBLAS=on -DCMAKE_CUDA_ARCHITECTURES=all-major" FORCE_CMAKE=1
pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade --verbose
```

It ran successfully and yielded llama.dll
However, when i try to load, it is throwing error

```
File "D:\My_Tech\GenAIPlay\venv\Lib\site-packages\llama_cpp\llama_cpp.py", line 72, in _load_shared_library
raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}")
RuntimeError: Failed to load shared library 'D:\My_Tech\GenAIPlay\venv\Lib\site-packages\llama_cpp\llama.dll': Could not find module 'D:\My_Tech\GenAIPlay\venv\Lib\site-packages\llama_cpp\llama.dll' (or one of its dependencies). Try using the full path with constructor syntax.
```
I tried fixing it as per other suggestions by modifying llama_cpp.py where the error was throwing as below but didn't work

return ctypes.CDLL(str(_lib_path),winmode=0)

I did set environment variables as well, still couldn't work.
Can you please help how can i fix it ?
```
$ echo $CUDA_PATH
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4
(venv)
umapa@UMA-SB2 MINGW64 /d/My_Tech/GenAIPlay
$ echo $PATH
D:\My_Tech\GenAIPlay\venv/Scripts:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\libnvvp;C:\Python311\Scripts\;C:\Python31
1\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Users\umapa\AppData\Local\Packages\PythonSoftwareFoundatio
n.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\Scripts;C:\Program Files\Git\cmd;C:\Program Files\Docker\Docker\resources\bin;C:\Program Files\nodejs\;C:\ProgramData\chocolatey\bi
n;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files\NVIDIA Corporation\Nsight Compute 2024.1.0\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Prog
ram Files (x86)\Incredibuild;C:\Users\umapa\AppData\Local\Microsoft\WindowsApps;C:\Users\umapa\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages
\Python39\Scripts;;C:\Users\umapa\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\umapa\AppData\Roaming\npm;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4\lib;D:\My_Tech\GenAIPlay\venv\Lib\site-packages\llama_cpp
```

```INSTALLATION NOTES
*** Installing project into wheel...
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/ggml_shared.lib
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/ggml_shared.dll
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/cmake/Llama/LlamaConfig.cmake
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/cmake/Llama/LlamaConfigVersion.cmake
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/ggml.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/ggml-alloc.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/ggml-backend.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/ggml-cuda.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/llama.lib
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/llama.dll
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/include/llama.h
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/convert.py
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/convert-lora-to-ggml.py
-- Installing: C:/Users/umapa/AppData/Local/Temp/tmpffhzeu6k/wheel/platlib/llama_cpp/llama.lib
-- Installing: C:/Users/umapa/AppData/Local/Temp/tmpffhzeu6k/wheel/platlib/llama_cpp/llama.dll
-- Installing: C:/Users/umapa/AppData/Local/Temp/pip-install-cn0mqw5n/llama-cpp-python_bd51aa929f42429a9180b8d6bd519841/llama_cpp/llama.lib
-- Installing: C:/Users/umapa/AppData/Local/Temp/pip-install-cn0mqw5n/llama-cpp-python_bd51aa929f42429a9180b8d6bd519841/llama_cpp/llama.dll
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/lib/llava.lib
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/llava.dll
-- Installing: C:\Users\umapa\AppData\Local\Temp\tmpffhzeu6k\wheel\platlib/bin/llava-cli.exe
-- Installing: C:/Users/umapa/AppData/Local/Temp/tmpffhzeu6k/wheel/platlib/llama_cpp/llava.lib
-- Installing: C:/Users/umapa/AppData/Local/Temp/tmpffhzeu6k/wheel/platlib/llama_cpp/llava.dll
-- Installing: C:/Users/umapa/AppData/Local/Temp/pip-install-cn0mqw5n/llama-cpp-python_bd51aa929f42429a9180b8d6bd519841/llama_cpp/llava.lib
-- Installing: C:/Users/umapa/AppData/Local/Temp/pip-install-cn0mqw5n/llama-cpp-python_bd51aa929f42429a9180b8d6bd519841/llama_cpp/llava.dll
*** Making wheel...
*** Created llama_cpp_python-0.2.56-cp311-cp311-win_amd64.whl...
Building wheel for llama-cpp-python (pyproject.toml) ... done
Created wheel for llama-cpp-python: filename=llama_cpp_python-0.2.56-cp311-cp311-win_amd64.whl size=22345276 sha256=ffffb1a35fc1e2b8a49a1c90d0e4f4a490ed71c298a29cd468816bbb6251aad7
Stored in directory: C:\Users\umapa\AppData\Local\Temp\pip-ephem-wheel-cache-px49kwop\wheels\f5\48\62\014b1a3c38f77df21219f81ed63ca4c09531d52a205b15d8e4
Successfully built llama-cpp-python
Installing collected packages: typing-extensions, numpy, MarkupSafe, diskcache, jinja2, llama-cpp-python
```

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.