AlexsJones / AlexsJones/llmfit
(FEAT) Ability to find ideal inference engine
未關閉
Automation
Efficiency
enhancement
- 主要語言
- Rust
- 星號
- 36.3k
- 分支
- 2.3k
- 平均合併
- 2 天 18 小時
- 30 天內合併 PR
- 85
描述
I know it's big feature request, but if it can be done it will benefit community immensely.
Basically figuring out automatically which inference engine+settings+quantization for any model is going to be ideal for system.
Will need going inference engines one by one, which will be lot of work. But thing is, lot of people do it manually countless times. If it can be automated, it will save countless individual hours.
貢獻指南
評估
這個 Issue 還沒有評估資料。