abetlen / abetlen/llama-cpp-python
BUG: import error or execute error (NULL pointer access) for the latest prebuilt version `v0.2.81`
- Langage dominant
- Python
- Étoiles
- 10.6k
- Forks
- 1.4k
- Métriques de merge des PR
- Métriques de PR en attente
Description
For the latest version `v0.2.81`:
1. If I install it via prebuilt channel:
```
pip install -U llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
```
Import error happens:
```
from llama_cpp import Llama
```
Error:
```
In [1]: from llama_cpp import Llama
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
File ~/miniconda3/lib/python3.9/site-packages/llama_cpp/llama_cpp.py:75, in _load_shared_library(lib_base_name)
74 try:
---> 75 return ctypes.CDLL(str(_lib_path), **cdll_args) # type: ignore
76 except Exception as e:
File ~/miniconda3/lib/python3.9/ctypes/__init__.py:374, in CDLL.__init__(self, name, mode, handle, use_errno, use_last_error, winmode)
373 if handle is None:
--> 374 self._handle = _dlopen(self._name, mode)
375 else:
OSError: libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory
During handling of the above exception, another exception occurred:
RuntimeError Traceback (most recent call last)
Cell In[1], line 1
----> 1 from llama_cpp import Llama
File ~/miniconda3/lib/python3.9/site-packages/llama_cpp/__init__.py:1
----> 1 from .llama_cpp import *
2 from .llama import *
4 __version__ = "0.2.81"
File ~/miniconda3/lib/python3.9/site-packages/llama_cpp/llama_cpp.py:88
85 _lib_base_name = "llama"
87 # Load the library
---> 88 _lib = _load_shared_library(_lib_base_name)
91 # ctypes sane type hint helpers
92 #
93 # - Generic Pointer and Array types
(...)
96 # NOTE: Only use these for static type checking not for runtime checks
97 # no good will come of that
99 if TYPE_CHECKING:
File ~/miniconda3/lib/python3.9/site-packages/llama_cpp/llama_cpp.py:77, in _load_shared_library(lib_base_name)
75 return ctypes.CDLL(str(_lib_path), **cdll_args) # type: ignore
76 except Exception as e:
---> 77 raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}")
79 raise FileNotFoundError(
80 f"Shared library with base name '{lib_base_name}' not found"
81 )
RuntimeError: Failed to load shared library '/home/lichengjie/miniconda3/lib/python3.9/site-packages/llama_cpp/lib/libllama.so': libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory
```
2. I install prebuilt version in `Docker`, the import can be passed successfully. However, the `NULL pointer access` happend when executing inference.
The error stack:
```
File "/opt/conda/lib/python3.11/site-packages/llama_cpp/llama.py", line 1132, in _create_completion
for token in self.generate(
^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/llama_cpp/llama.py", line 740, in generate
self.eval(tokens)
File "/opt/conda/lib/python3.11/site-packages/llama_cpp/llama.py", line 590, in eval
logits = np.ctypeslib.as_array(self._ctx.get_logits(), shape=(rows * cols, ))
^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/numpy/ctypeslib.py", line 522, in as_array
obj = ctypes.cast(obj, p_arr_type).contents
^^^^^^^^^^^^^^^^^
ValueError: [address=0.0.0.0:37293, pid=88] NULL pointer access
```
I tried prebuilt `v0.2.77`, everything is ok.
Could you please provide some solutions for these issues? Thank you.
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.