conda-forge / conda-forge/llama.cpp-feedstock

Use `GGML_BACKEND_DL` and `GGML_CPU_ALL_VARIANTS` for x86_64 builds when it's ready

Open
#68 1 comment 2 reactions 0 assignees View on GitHub
question
Dominant language
No language data
Stars
6
Forks
18
Avg merge
13h 42m
Merged PRs (30d)
4

Description

### Comment:

Hi maintainers!

There was a cool feature added to llama.cpp where we can have dynamic dispatch to various backends which are compiled for the various microarchitecture levels for x86_64. This can enabled great performance benefits! It's basically enabling `GGML_BACKEND_DL` and `GGML_CPU_ALL_VARIANTS`.

I created an issue for llama-cpp-python to add some missing bindings to really make use of this, see here https://github.com/abetlen/llama-cpp-python/issues/2069

When this has been taken care of and llama-cpp-python supports this, I think it would be really great if this feedstock would enable this for the x86_64 CPU builds.

Contributor guide

Open the contributing guide

Research direction

Start by checking the linked llama-cpp-python issue 2069 and the feedstock's current x86_64 CPU build configuration. When the missing bindings are available, enable GGML_BACKEND_DL and GGML_CPU_ALL_VARIANTS for those builds and confirm that the feedstock builds successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
build-system
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.