AlexsJones / AlexsJones/llmfit

(FEAT) Ability to find ideal inference engine

未關閉
#80 4 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
Automation Efficiency enhancement
主要語言
Rust
星號
36.3k
分支
2.3k
平均合併
2 天 18 小時
30 天內合併 PR
85

描述

I know it's big feature request, but if it can be done it will benefit community immensely.
Basically figuring out automatically which inference engine+settings+quantization for any model is going to be ideal for system.
Will need going inference engines one by one, which will be lot of work. But thing is, lot of people do it manually countless times. If it can be automated, it will save countless individual hours.

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。