Inference speed on windows server 2022 which has Nvidia RTX A4000

未關閉

#973 建立於 2023年12月5日 4 則留言 0 個 reaction 已指派 0 人在 GitHub 檢視

bug performance

描述

i am doing mistral 7b openorca inference using llamacpp-python but its is taken lot of timing.How can i fix that

llama-cpp-python version is 0.2.11
Server Configuration:
1)Windows Server 2022 Standard
2) two Nvidia rtx A4000 gpu

![image](https://github.com/abetlen/llama-cpp-python/assets/89073843/43e63de2-9298-4805-b5be-069d36a8157c)

這個 Issue 還沒有評估資料。