abetlen / abetlen/llama-cpp-python

BUG: import error or execute error (NULL pointer access) for the latest prebuilt version `v0.2.81`

Đang mở
#1,571 8 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
10.6k
Fork
1.4k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

For the latest version `v0.2.81`:

1. If I install it via prebuilt channel:
```
pip install -U llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
```
Import error happens:
```
from llama_cpp import Llama
```
Error:
```
In [1]: from llama_cpp import Llama
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
File ~/miniconda3/lib/python3.9/site-packages/llama_cpp/llama_cpp.py:75, in _load_shared_library(lib_base_name)
74 try:
---> 75 return ctypes.CDLL(str(_lib_path), **cdll_args) # type: ignore
76 except Exception as e:

File ~/miniconda3/lib/python3.9/ctypes/__init__.py:374, in CDLL.__init__(self, name, mode, handle, use_errno, use_last_error, winmode)
373 if handle is None:
--> 374 self._handle = _dlopen(self._name, mode)
375 else:

OSError: libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

RuntimeError Traceback (most recent call last)
Cell In[1], line 1
----> 1 from llama_cpp import Llama

File ~/miniconda3/lib/python3.9/site-packages/llama_cpp/__init__.py:1
----> 1 from .llama_cpp import *
2 from .llama import *
4 __version__ = "0.2.81"

File ~/miniconda3/lib/python3.9/site-packages/llama_cpp/llama_cpp.py:88
85 _lib_base_name = "llama"
87 # Load the library
---> 88 _lib = _load_shared_library(_lib_base_name)
91 # ctypes sane type hint helpers
92 #
93 # - Generic Pointer and Array types
(...)
96 # NOTE: Only use these for static type checking not for runtime checks
97 # no good will come of that
99 if TYPE_CHECKING:

File ~/miniconda3/lib/python3.9/site-packages/llama_cpp/llama_cpp.py:77, in _load_shared_library(lib_base_name)
75 return ctypes.CDLL(str(_lib_path), **cdll_args) # type: ignore
76 except Exception as e:
---> 77 raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}")
79 raise FileNotFoundError(
80 f"Shared library with base name '{lib_base_name}' not found"
81 )

RuntimeError: Failed to load shared library '/home/lichengjie/miniconda3/lib/python3.9/site-packages/llama_cpp/lib/libllama.so': libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory
```

2. I install prebuilt version in `Docker`, the import can be passed successfully. However, the `NULL pointer access` happend when executing inference.
The error stack:
```
File "/opt/conda/lib/python3.11/site-packages/llama_cpp/llama.py", line 1132, in _create_completion
for token in self.generate(
^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/llama_cpp/llama.py", line 740, in generate
self.eval(tokens)
File "/opt/conda/lib/python3.11/site-packages/llama_cpp/llama.py", line 590, in eval
logits = np.ctypeslib.as_array(self._ctx.get_logits(), shape=(rows * cols, ))
^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/numpy/ctypeslib.py", line 522, in as_array
obj = ctypes.cast(obj, p_arr_type).contents
^^^^^^^^^^^^^^^^^
ValueError: [address=0.0.0.0:37293, pid=88] NULL pointer access
```

I tried prebuilt `v0.2.77`, everything is ok.
Could you please provide some solutions for these issues? Thank you.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.