abetlen / abetlen/llama-cpp-python

Unable to activate the Cublas while running a python file which implements the llamachain using the llamacpp

Đang mở
#658 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
build
Ngôn ngữ chính
Python
Star
10.6k
Fork
1.4k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

I was trying to activate the Blas backend since a month to improve the performance by using GPU, and finally was able to activate the cublas that uses the GPU while running the main.exe from the llama.cpp/build/bin/Release following these steps : [https://github.com/ggerganov/llama.cpp#cublas](https://github.com/ggerganov/llama.cpp/issues/url)

I have been running a LlamaChain that implements Llamacpp inside a python file, where in the model properties it's showing

BLAS = 0

and thus not using the GPU.

Required : I want to activate the CuBlas upon running this python file and utilize the GPU to improve the performance. Any assistance on this issue is highly appreciated.
Thanks in advance.

System specifications :
OS : Windows 11
Blas used : CuBLAS
GPU : NVIDIA T600 Laptop GPU

Thanks [staviq](https://github.com/staviq) for redirecting here :) Duplicate issue : https://github.com/ggerganov/llama.cpp/issues/2958

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.