abetlen / abetlen/llama-cpp-python

Llama-cpp-python installation issues on MacOS with M2

Abierto
#956 5 comentarios 0 reacciones 0 asignados Ver en GitHub
bug build
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

I am following the instructions found in the file macOS.md. Everything worked fine until I get to the line:
```
pip install 'llama-cpp-python[server]'
```
which also installs correctly. First, the notation `[server]` is new to me. Is this some kind of standard?

Second, I follow the instructions and type:
```
export MODEL=mistral-7b-instruct-v0.1.Q3_K_M.gguf
python3 -m llama_cpp.server --model $MODEL --n_gpu_layers 1
```
and get the message:
```
Traceback (most recent call last):
File "/Users/erlebach/miniconda3/envs/llama/lib/python3.9/runpy.py", line 188, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/Users/erlebach/miniconda3/envs/llama/lib/python3.9/runpy.py", line 111, in _get_module_details
__import__(pkg_name)
File "/Users/erlebach/src/2023/llama-cpp-python/llama_cpp/__init__.py", line 1, in
from .llama_cpp import *
File "/Users/erlebach/src/2023/llama-cpp-python/llama_cpp/llama_cpp.py", line 82, in
_lib = _load_shared_library(_lib_base_name)
File "/Users/erlebach/src/2023/llama-cpp-python/llama_cpp/llama_cpp.py", line 73, in _load_shared_library
raise FileNotFoundError(
FileNotFoundError: Shared library with base name 'llama' not found
```
This error seems to be unrelated to the model. What could be the problem? Thanks.

(Could the error be due to the fact that I am running a Q3_K_M model, and not a Q4_K_M model? Why should that be?
Thanks!

Gordon

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.