abetlen / abetlen/llama-cpp-python

BUG: import error or execute error (NULL pointer access) for the latest prebuilt version `v0.2.81`

Offen
#1,571 8 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
10.6k
Forks
1.4k
PR-Merge-Kennzahlen
PR-Kennzahlen ausstehend

Beschreibung

For the latest version `v0.2.81`:

1. If I install it via prebuilt channel:
```
pip install -U llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
```
Import error happens:
```
from llama_cpp import Llama
```
Error:
```
In [1]: from llama_cpp import Llama
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
File ~/miniconda3/lib/python3.9/site-packages/llama_cpp/llama_cpp.py:75, in _load_shared_library(lib_base_name)
74 try:
---> 75 return ctypes.CDLL(str(_lib_path), **cdll_args) # type: ignore
76 except Exception as e:

File ~/miniconda3/lib/python3.9/ctypes/__init__.py:374, in CDLL.__init__(self, name, mode, handle, use_errno, use_last_error, winmode)
373 if handle is None:
--> 374 self._handle = _dlopen(self._name, mode)
375 else:

OSError: libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

RuntimeError Traceback (most recent call last)
Cell In[1], line 1
----> 1 from llama_cpp import Llama

File ~/miniconda3/lib/python3.9/site-packages/llama_cpp/__init__.py:1
----> 1 from .llama_cpp import *
2 from .llama import *
4 __version__ = "0.2.81"

File ~/miniconda3/lib/python3.9/site-packages/llama_cpp/llama_cpp.py:88
85 _lib_base_name = "llama"
87 # Load the library
---> 88 _lib = _load_shared_library(_lib_base_name)
91 # ctypes sane type hint helpers
92 #
93 # - Generic Pointer and Array types
(...)
96 # NOTE: Only use these for static type checking not for runtime checks
97 # no good will come of that
99 if TYPE_CHECKING:

File ~/miniconda3/lib/python3.9/site-packages/llama_cpp/llama_cpp.py:77, in _load_shared_library(lib_base_name)
75 return ctypes.CDLL(str(_lib_path), **cdll_args) # type: ignore
76 except Exception as e:
---> 77 raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}")
79 raise FileNotFoundError(
80 f"Shared library with base name '{lib_base_name}' not found"
81 )

RuntimeError: Failed to load shared library '/home/lichengjie/miniconda3/lib/python3.9/site-packages/llama_cpp/lib/libllama.so': libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory
```

2. I install prebuilt version in `Docker`, the import can be passed successfully. However, the `NULL pointer access` happend when executing inference.
The error stack:
```
File "/opt/conda/lib/python3.11/site-packages/llama_cpp/llama.py", line 1132, in _create_completion
for token in self.generate(
^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/llama_cpp/llama.py", line 740, in generate
self.eval(tokens)
File "/opt/conda/lib/python3.11/site-packages/llama_cpp/llama.py", line 590, in eval
logits = np.ctypeslib.as_array(self._ctx.get_logits(), shape=(rows * cols, ))
^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/numpy/ctypeslib.py", line 522, in as_array
obj = ctypes.cast(obj, p_arr_type).contents
^^^^^^^^^^^^^^^^^
ValueError: [address=0.0.0.0:37293, pid=88] NULL pointer access
```

I tried prebuilt `v0.2.77`, everything is ok.
Could you please provide some solutions for these issues? Thank you.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.