abetlen / abetlen/llama-cpp-python

Problem with importing llama-cpp-python library on Linux CentOS

Aberta
#1,099 2 comentários 1 reação 0 responsáveis Ver no GitHub
bug
Linguagem predominante
Python
Estrelas
10.6k
Forks
1.4k
Métricas de merge de PRs
Métricas de PR pendentes

Descrição

It's my first bug report on the GitHub, please, don't be mad if I've done something wrong.

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [☑️] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [☑️] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [☑️] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [☑️] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior

I'm new to LLM and wanted to try it locally. So I found hugging_face and langchain, which I used in my implementation then. So I expected that there would be no problem with the import, especially after making sure that the llama-cpp-python library, which is used in mentioned frameworks, was installed successfully.

# Current Behavior

But every time when compiler comes to the line with LlamaCpp object I got error **ImportError: Could not import llama-cpp-python library. Please install the llama-cpp-python library to use this embedding model: pip install llama-cpp-python**. I tried to upgrade it with following command `pip install llama-cpp-python --upgrade --force-reinstall --no-cache-dir`, but it didn't help.

# Environment and Context

* I am using Virtual Machine with following properties:

CentOS Linux release 8.5.2111

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel Xeon Processor (Cascadelake)
Stepping: 5
CPU MHz: 2095.076
BogoMIPS: 4190.15
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 4096K
L3 cache: 16384K
NUMA node0 CPU(s): 0-7

```
Python 3.10.9
GNU Make 4.2.1
g++ (GCC) 8.5.0 20210514 (Red Hat 8.5.0-4)
langchain==0.1.1
langchain-community==0.0.12
llama-cpp-python==0.2.28
```

# Failure Information (for bugs)
**ImportError: Could not import llama-cpp-python library. Please install the llama-cpp-python library to use this embedding model: pip install llama-cpp-python**

# Steps to Reproduce

1. Create venv with Python 3.10.9
2. Import LlamaCpp through langchain_community or langchain. Import hf_hub_download from huggingface_hub
3. Declare

downloaded_model_path = hf_hub_download(repo_id="TheBloke/Mistral-7B-Instruct-v0.1-GGUF",
filename="mistral-7b-instruct-v0.1.Q4_0.gguf")

llm = LlamaCpp(
model_path=downloaded_model_path,
n_ctx=2048,
n_threads=10,
n_gpu_layers=25,
temp=0.1,
n_batch = 512,
n_predict = -1,
n_keep = 0
)
5. Get the error.

**Note: Many issues seem to be regarding functional or performance issues / differences with `llama.cpp`. In these cases we need to confirm that you're comparing against the version of `llama.cpp` that was built with your python package, and which parameters you're passing to the context.**

Try the following:

1. `git clone https://github.com/abetlen/llama-cpp-python`
2. `cd llama-cpp-python`
3. `rm -rf _skbuild/` # delete any old builds
5. `python -m pip install .`
6. `cd ./vendor/llama.cpp`
7. Follow [llama.cpp's instructions](https://github.com/ggerganov/llama.cpp#build) to `cmake` llama.cpp *!Errors on this step! Seems like I need CMake >3.21, and mine is 3.20.2. Encounter errors with `./bootstrap` while upgrading. Can't resolve it...*
9. Run llama.cpp's `./main` with the same arguments you previously passed to llama-cpp-python and see if you can reproduce the issue. If you can, [log an issue with llama.cpp](https://github.com/ggerganov/llama.cpp/issues)

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.