abetlen / abetlen/llama-cpp-python

cuBLAS error 15

Abierto
#941 2 comentarios 1 reacción 0 asignados Ver en GitHub
bug
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [x] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [x] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [x] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior

I am trying to run [privateGPT](https://github.com/imartinez/privateGPT) with CUDA.
Without CUDA it works. With CUDA enabled, 'llama-cpp-python' crashes.

# Current Behavior
When I start privateGPT, llama-cpp-python-main gives this error : ```cuBLAS error 15 at C:\Users\Harry\Documents\llama-cpp-python-main\vendor\llama.cpp\ggml-cuda.cu:7586: the requested functionality is not supported```

# Environment and Context

I am on Windows 11, with NVIDIA CUDA Toolkit 12.2 installed, NVIDIA GeForce GTX 960 4GB and 32GB RAM. CPU is Intel i7 2600K (it should support AVX).

```
$ python3 --version
$ Python 3.11.6

$ make --version
$ GNU Make 4.4.1

$ g++ --version
$ g++ (x86_64-posix-seh-rev0, Built by MinGW-Builds project) 13.2.0
```

# Failure Information (for bugs)
```cuBLAS error 15 at C:\Users\Harry\Documents\llama-cpp-python-main\vendor\llama.cpp\ggml-cuda.cu:7586: the requested functionality is not supported```

# Steps to Reproduce

```
git clone https://github.com/imartinez/privateGPT
cd privateGPT

conda create -n privateGPT python=3.11
conda activate privateGPT

poetry install --with ui,local

poetry run python scripts/setup

cd ..

git clone --recurse-submodules https://github.com/abetlen/llama-cpp-python.git llama-cpp-python-main
cd llama-cpp-python-main

set FORCE_CMAKE=1
set CMAKE_ARGS=-DLLAMA_CUBLAS=on -DLLAMA_AVX=on -DLLAMA_AVX2=off -DLLAMA_FMA=off
python -m pip install .[all]

cd ..
cd privateGPT
make run
```

Let me know if I can give you more info, thank you.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.