abetlen / abetlen/llama-cpp-python
Multi-arch support for pre-built cpu wheel
Offen
enhancement
- Vorherrschende Sprache
- Python
- Sterne
- 10.6k
- Forks
- 1.4k
- PR-Merge-Kennzahlen
- PR-Kennzahlen ausstehend
Beschreibung
Currently pre-built cpu wheels are compiled for the lowest common denominator architecture. If we can compile multiple versions of the llama.cpp library with different accelerations compiled in each we can bundle all of them into the pre-built wheels and dynamically choose one based on the host cpu.
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.