abetlen / abetlen/llama-cpp-python
Can RotorQuant/TurboQuant and dflash support be added?
Ouverte
- Langage dominant
- Python
- Étoiles
- 10.6k
- Forks
- 1.4k
- Métriques de merge des PR
- Métriques de PR en attente
Description
Hi!
This project is very useful for using with llama.cpp in python. However, it would be great if we could have some features even before the main llama.cpp has full support for them. By that I mean TurboQuant/RotorQuant and dFlash speculative decoding. It would make this entire library AMAZING to use.
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.