abetlen / abetlen/llama-cpp-python

Llama-cpp-python installation issues on MacOS with M2

Đang mở
#956 5 bình luận 0 reaction 0 người được giao Xem trên GitHub
bug build
Ngôn ngữ chính
Python
Star
10.6k
Fork
1.4k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

I am following the instructions found in the file macOS.md. Everything worked fine until I get to the line:
```
pip install 'llama-cpp-python[server]'
```
which also installs correctly. First, the notation `[server]` is new to me. Is this some kind of standard?

Second, I follow the instructions and type:
```
export MODEL=mistral-7b-instruct-v0.1.Q3_K_M.gguf
python3 -m llama_cpp.server --model $MODEL --n_gpu_layers 1
```
and get the message:
```
Traceback (most recent call last):
File "/Users/erlebach/miniconda3/envs/llama/lib/python3.9/runpy.py", line 188, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/Users/erlebach/miniconda3/envs/llama/lib/python3.9/runpy.py", line 111, in _get_module_details
__import__(pkg_name)
File "/Users/erlebach/src/2023/llama-cpp-python/llama_cpp/__init__.py", line 1, in
from .llama_cpp import *
File "/Users/erlebach/src/2023/llama-cpp-python/llama_cpp/llama_cpp.py", line 82, in
_lib = _load_shared_library(_lib_base_name)
File "/Users/erlebach/src/2023/llama-cpp-python/llama_cpp/llama_cpp.py", line 73, in _load_shared_library
raise FileNotFoundError(
FileNotFoundError: Shared library with base name 'llama' not found
```
This error seems to be unrelated to the model. What could be the problem? Thanks.

(Could the error be due to the fact that I am running a Q3_K_M model, and not a Q4_K_M model? Why should that be?
Thanks!

Gordon

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.