abetlen / abetlen/llama-cpp-python

Llama-cpp-python installation issues on MacOS with M2

未關閉
#956 5 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
bug build
主要語言
Python
星號
10.6k
分支
1.4k
PR 合併指標
PR 指標待擷取

描述

I am following the instructions found in the file macOS.md. Everything worked fine until I get to the line:
```
pip install 'llama-cpp-python[server]'
```
which also installs correctly. First, the notation `[server]` is new to me. Is this some kind of standard?

Second, I follow the instructions and type:
```
export MODEL=mistral-7b-instruct-v0.1.Q3_K_M.gguf
python3 -m llama_cpp.server --model $MODEL --n_gpu_layers 1
```
and get the message:
```
Traceback (most recent call last):
File "/Users/erlebach/miniconda3/envs/llama/lib/python3.9/runpy.py", line 188, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/Users/erlebach/miniconda3/envs/llama/lib/python3.9/runpy.py", line 111, in _get_module_details
__import__(pkg_name)
File "/Users/erlebach/src/2023/llama-cpp-python/llama_cpp/__init__.py", line 1, in
from .llama_cpp import *
File "/Users/erlebach/src/2023/llama-cpp-python/llama_cpp/llama_cpp.py", line 82, in
_lib = _load_shared_library(_lib_base_name)
File "/Users/erlebach/src/2023/llama-cpp-python/llama_cpp/llama_cpp.py", line 73, in _load_shared_library
raise FileNotFoundError(
FileNotFoundError: Shared library with base name 'llama' not found
```
This error seems to be unrelated to the model. What could be the problem? Thanks.

(Could the error be due to the fact that I am running a Q3_K_M model, and not a Q4_K_M model? Why should that be?
Thanks!

Gordon

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。