abetlen / abetlen/llama-cpp-python
Problem with importing llama-cpp-python library on Linux CentOS
- 主要语言
- Python
- 星标
- 10.6k
- 派生
- 1.4k
- PR 合并指标
- PR 指标待抓取
描述
It's my first bug report on the GitHub, please, don't be mad if I've done something wrong.
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [☑️] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [☑️] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [☑️] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [☑️] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.
# Expected Behavior
I'm new to LLM and wanted to try it locally. So I found hugging_face and langchain, which I used in my implementation then. So I expected that there would be no problem with the import, especially after making sure that the llama-cpp-python library, which is used in mentioned frameworks, was installed successfully.
# Current Behavior
But every time when compiler comes to the line with LlamaCpp object I got error **ImportError: Could not import llama-cpp-python library. Please install the llama-cpp-python library to use this embedding model: pip install llama-cpp-python**. I tried to upgrade it with following command `pip install llama-cpp-python --upgrade --force-reinstall --no-cache-dir`, but it didn't help.
# Environment and Context
* I am using Virtual Machine with following properties:
CentOS Linux release 8.5.2111
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel Xeon Processor (Cascadelake)
Stepping: 5
CPU MHz: 2095.076
BogoMIPS: 4190.15
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 4096K
L3 cache: 16384K
NUMA node0 CPU(s): 0-7
```
Python 3.10.9
GNU Make 4.2.1
g++ (GCC) 8.5.0 20210514 (Red Hat 8.5.0-4)
langchain==0.1.1
langchain-community==0.0.12
llama-cpp-python==0.2.28
```
# Failure Information (for bugs)
**ImportError: Could not import llama-cpp-python library. Please install the llama-cpp-python library to use this embedding model: pip install llama-cpp-python**
# Steps to Reproduce
1. Create venv with Python 3.10.9
2. Import LlamaCpp through langchain_community or langchain. Import hf_hub_download from huggingface_hub
3. Declare
downloaded_model_path = hf_hub_download(repo_id="TheBloke/Mistral-7B-Instruct-v0.1-GGUF",
filename="mistral-7b-instruct-v0.1.Q4_0.gguf")
llm = LlamaCpp(
model_path=downloaded_model_path,
n_ctx=2048,
n_threads=10,
n_gpu_layers=25,
temp=0.1,
n_batch = 512,
n_predict = -1,
n_keep = 0
)
5. Get the error.
**Note: Many issues seem to be regarding functional or performance issues / differences with `llama.cpp`. In these cases we need to confirm that you're comparing against the version of `llama.cpp` that was built with your python package, and which parameters you're passing to the context.**
Try the following:
1. `git clone https://github.com/abetlen/llama-cpp-python`
2. `cd llama-cpp-python`
3. `rm -rf _skbuild/` # delete any old builds
5. `python -m pip install .`
6. `cd ./vendor/llama.cpp`
7. Follow [llama.cpp's instructions](https://github.com/ggerganov/llama.cpp#build) to `cmake` llama.cpp *!Errors on this step! Seems like I need CMake >3.21, and mine is 3.20.2. Encounter errors with `./bootstrap` while upgrading. Can't resolve it...*
9. Run llama.cpp's `./main` with the same arguments you previously passed to llama-cpp-python and see if you can reproduce the issue. If you can, [log an issue with llama.cpp](https://github.com/ggerganov/llama.cpp/issues)
贡献指南
评估
这个 Issue 还没有评估数据。