abetlen / abetlen/llama-cpp-python
Can RotorQuant/TurboQuant and dflash support be added?
Aperta
- Lingua principale
- Python
- Stelle
- 10.6k
- Fork
- 1.4k
- Metriche di merge delle PR
- Metriche PR in attesa
Descrizione
Hi!
This project is very useful for using with llama.cpp in python. However, it would be great if we could have some features even before the main llama.cpp has full support for them. By that I mean TurboQuant/RotorQuant and dFlash speculative decoding. It would make this entire library AMAZING to use.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.