abetlen / abetlen/llama-cpp-python
Llama-cpp-python installation issues on MacOS with M2
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 1.4k
- PR merge metrics
- PR metrics pending
Description
I am following the instructions found in the file macOS.md. Everything worked fine until I get to the line:
```
pip install 'llama-cpp-python[server]'
```
which also installs correctly. First, the notation `[server]` is new to me. Is this some kind of standard?
Second, I follow the instructions and type:
```
export MODEL=mistral-7b-instruct-v0.1.Q3_K_M.gguf
python3 -m llama_cpp.server --model $MODEL --n_gpu_layers 1
```
and get the message:
```
Traceback (most recent call last):
File "/Users/erlebach/miniconda3/envs/llama/lib/python3.9/runpy.py", line 188, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/Users/erlebach/miniconda3/envs/llama/lib/python3.9/runpy.py", line 111, in _get_module_details
__import__(pkg_name)
File "/Users/erlebach/src/2023/llama-cpp-python/llama_cpp/__init__.py", line 1, in
from .llama_cpp import *
File "/Users/erlebach/src/2023/llama-cpp-python/llama_cpp/llama_cpp.py", line 82, in
_lib = _load_shared_library(_lib_base_name)
File "/Users/erlebach/src/2023/llama-cpp-python/llama_cpp/llama_cpp.py", line 73, in _load_shared_library
raise FileNotFoundError(
FileNotFoundError: Shared library with base name 'llama' not found
```
This error seems to be unrelated to the model. What could be the problem? Thanks.
(Could the error be due to the fact that I am running a Q3_K_M model, and not a Q4_K_M model? Why should that be?
Thanks!
Gordon
Contributor guide
Assessment
This issue has not been assessed yet.