abetlen / abetlen/llama-cpp-python
Can RotorQuant/TurboQuant and dflash support be added?
Đang mở
- Ngôn ngữ chính
- Python
- Star
- 10.6k
- Fork
- 1.4k
- Chỉ số merge pull request
- Chỉ số pull request đang chờ
Mô tả
Hi!
This project is very useful for using with llama.cpp in python. However, it would be great if we could have some features even before the main llama.cpp has full support for them. By that I mean TurboQuant/RotorQuant and dFlash speculative decoding. It would make this entire library AMAZING to use.
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.