abetlen / abetlen/llama-cpp-python

Intel Arc Support?

Đang mở
#901 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
enhancement question
Ngôn ngữ chính
Python
Star
10.6k
Fork
1.4k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

Hello there, I was wondering if it would be possible to add support for Intel ARC GPUs?

I know it's possible with llama.cpp as I've managed to do it through compiling CLBlast via oneAPI's C compiler, and then using that with oneAPI's C compiler in path to build llama.cpp and it successfully, the only thing other than that is the right platform needs selecting as an option when launching else it will attempt to use Intel's emulation for OpenCL instead.

I've tested this version of llama.cpp and you can successfully offload layers to the GPU and the GPU is then utilised as well (looking and vram and tokens/s).

If you would like me to complete the compilations for you every time you want to update the project and contribute them, I could also do that. I would just need the .CPP files that you are embedding into your python code.

Many thanks,
Elliott Dyson

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.