abetlen / abetlen/llama-cpp-python

Using a pre-built wheel currently requires specifying the right version, e.g. llama-cpp-python==0.3.4

Open
#1,872 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
1.4k
PR merge metrics
PR metrics pending

Description

Ran into this problem, and it might be helpful for others if this were clarified in the help text. When I run pip install llama-cpp-python it tries to use 0.3.5, and if that version isn't present in the extra-index-url, it will just ignore it and still try to build the wheel itself.

e.g. for CUDA it reccomends

```
pip install llama-cpp-python \
--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
```
but it only worked for me after clicking into the link and seeing that it only supports 0.3.4 / 0.2.9 / ...

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.