abetlen / abetlen/llama-cpp-python
Can RotorQuant/TurboQuant and dflash support be added?
未关闭
- 主要语言
- Python
- 星标
- 10.6k
- 派生
- 1.4k
- PR 合并指标
- PR 指标待抓取
描述
Hi!
This project is very useful for using with llama.cpp in python. However, it would be great if we could have some features even before the main llama.cpp has full support for them. By that I mean TurboQuant/RotorQuant and dFlash speculative decoding. It would make this entire library AMAZING to use.
贡献指南
评估
这个 Issue 还没有评估数据。