abetlen / abetlen/llama-cpp-python

cuBLAS error 15

Ouverte
#941 2 commentaires 1 réaction 0 personnes assignées Voir sur GitHub
bug
Langage dominant
Python
Étoiles
10.6k
Forks
1.4k
Métriques de merge des PR
Métriques de PR en attente

Description

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [x] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [x] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [x] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior

I am trying to run [privateGPT](https://github.com/imartinez/privateGPT) with CUDA.
Without CUDA it works. With CUDA enabled, 'llama-cpp-python' crashes.

# Current Behavior
When I start privateGPT, llama-cpp-python-main gives this error : ```cuBLAS error 15 at C:\Users\Harry\Documents\llama-cpp-python-main\vendor\llama.cpp\ggml-cuda.cu:7586: the requested functionality is not supported```

# Environment and Context

I am on Windows 11, with NVIDIA CUDA Toolkit 12.2 installed, NVIDIA GeForce GTX 960 4GB and 32GB RAM. CPU is Intel i7 2600K (it should support AVX).

```
$ python3 --version
$ Python 3.11.6

$ make --version
$ GNU Make 4.4.1

$ g++ --version
$ g++ (x86_64-posix-seh-rev0, Built by MinGW-Builds project) 13.2.0
```

# Failure Information (for bugs)
```cuBLAS error 15 at C:\Users\Harry\Documents\llama-cpp-python-main\vendor\llama.cpp\ggml-cuda.cu:7586: the requested functionality is not supported```

# Steps to Reproduce

```
git clone https://github.com/imartinez/privateGPT
cd privateGPT

conda create -n privateGPT python=3.11
conda activate privateGPT

poetry install --with ui,local

poetry run python scripts/setup

cd ..

git clone --recurse-submodules https://github.com/abetlen/llama-cpp-python.git llama-cpp-python-main
cd llama-cpp-python-main

set FORCE_CMAKE=1
set CMAKE_ARGS=-DLLAMA_CUBLAS=on -DLLAMA_AVX=on -DLLAMA_AVX2=off -DLLAMA_FMA=off
python -m pip install .[all]

cd ..
cd privateGPT
make run
```

Let me know if I can give you more info, thank you.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.