conda-forge / conda-forge/llama.cpp-feedstock
Use `GGML_BACKEND_DL` and `GGML_CPU_ALL_VARIANTS` for x86_64 builds when it's ready
- Dominant language
- No language data
- Stars
- 6
- Forks
- 18
- Avg merge
- 13h 42m
- Merged PRs (30d)
- 4
Description
### Comment:
Hi maintainers!
There was a cool feature added to llama.cpp where we can have dynamic dispatch to various backends which are compiled for the various microarchitecture levels for x86_64. This can enabled great performance benefits! It's basically enabling `GGML_BACKEND_DL` and `GGML_CPU_ALL_VARIANTS`.
I created an issue for llama-cpp-python to add some missing bindings to really make use of this, see here https://github.com/abetlen/llama-cpp-python/issues/2069
When this has been taken care of and llama-cpp-python supports this, I think it would be really great if this feedstock would enable this for the x86_64 CPU builds.
Contributor guide
Research direction
Start by checking the linked llama-cpp-python issue 2069 and the feedstock's current x86_64 CPU build configuration. When the missing bindings are available, enable GGML_BACKEND_DL and GGML_CPU_ALL_VARIANTS for those builds and confirm that the feedstock builds successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100