abetlen / abetlen/llama-cpp-python

cuBLAS error 15

Đang mở
#941 2 bình luận 1 reaction 0 người được giao Xem trên GitHub
bug
Ngôn ngữ chính
Python
Star
10.6k
Fork
1.4k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [x] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [x] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [x] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior

I am trying to run [privateGPT](https://github.com/imartinez/privateGPT) with CUDA.
Without CUDA it works. With CUDA enabled, 'llama-cpp-python' crashes.

# Current Behavior
When I start privateGPT, llama-cpp-python-main gives this error : ```cuBLAS error 15 at C:\Users\Harry\Documents\llama-cpp-python-main\vendor\llama.cpp\ggml-cuda.cu:7586: the requested functionality is not supported```

# Environment and Context

I am on Windows 11, with NVIDIA CUDA Toolkit 12.2 installed, NVIDIA GeForce GTX 960 4GB and 32GB RAM. CPU is Intel i7 2600K (it should support AVX).

```
$ python3 --version
$ Python 3.11.6

$ make --version
$ GNU Make 4.4.1

$ g++ --version
$ g++ (x86_64-posix-seh-rev0, Built by MinGW-Builds project) 13.2.0
```

# Failure Information (for bugs)
```cuBLAS error 15 at C:\Users\Harry\Documents\llama-cpp-python-main\vendor\llama.cpp\ggml-cuda.cu:7586: the requested functionality is not supported```

# Steps to Reproduce

```
git clone https://github.com/imartinez/privateGPT
cd privateGPT

conda create -n privateGPT python=3.11
conda activate privateGPT

poetry install --with ui,local

poetry run python scripts/setup

cd ..

git clone --recurse-submodules https://github.com/abetlen/llama-cpp-python.git llama-cpp-python-main
cd llama-cpp-python-main

set FORCE_CMAKE=1
set CMAKE_ARGS=-DLLAMA_CUBLAS=on -DLLAMA_AVX=on -DLLAMA_AVX2=off -DLLAMA_FMA=off
python -m pip install .[all]

cd ..
cd privateGPT
make run
```

Let me know if I can give you more info, thank you.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.