abetlen / abetlen/llama-cpp-python

cu118 install falls back to CPU mode on CentOS 7 (0.3.8)

Open
#1,990 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
1.4k
PR merge metrics
PR metrics pending

Description

I'm using CentOS 7 (glibc 2.17) with both CUDA 11.8 and 12.4.

Up until version 0.3.7, I was able to install llama-cpp-python with either CUDA version. However, starting from 0.3.8, it seems that the cu118 variant is no longer supported, and the package installs in CPU mode instead.

Has anyone managed to get it working with a similar setup?

Here's the install command I'm using:

`FORCE_CMAKE=1 CMAKE_ARGS="-DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES='70;75'" pip install llama-cpp-python==0.3.8`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.