abetlen / abetlen/llama-cpp-python

Build Failure When Enabling KleidiAI on ARMv9 in llama-cpp-python ≥ 3.10.0

Abierto
#2,095 1 comentario 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

When I try to install the latest version of **llama-cpp-python** and enable **KleidiAI** on an **ARMv9 CPU**, I use the following command:

```bash
CMAKE_ARGS="-DGGML_NATIVE=OFF -DGGML_CPU_ARM_ARCH=armv9-a+i8mm+dotprod -DGGML_CPU_KLEIDIAI=ON" pip3 install llama-cpp-python
```

However, during the wheel-building process, it fails with the following error:

```bash
[93/94] : && /usr/bin/cmake -E rm -f vendor/llama.cpp/common/libcommon.a && /usr/bin/aarch64-linux-gnu-ar qc vendor/llama.cpp/common/libcommon.a vendor/llama.cpp/common/CMakeFiles/build_info.dir/build-info.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/arg.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/chat-parser.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/chat.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/common.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/console.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/json-partial.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/json-schema-to-grammar.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/llguidance.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/log.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/ngram-cache.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/regex-partial.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/sampling.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/speculative.cpp.o && /usr/bin/aarch64-linux-gnu-ranlib vendor/llama.cpp/common/libcommon.a && :
[94/94] : && /usr/bin/aarch64-linux-gnu-g++ -O3 -DNDEBUG vendor/llama.cpp/tools/mtmd/CMakeFiles/llama-mtmd-cli.dir/mtmd-cli.cpp.o -o vendor/llama.cpp/tools/mtmd/llama-mtmd-cli -Wl,-rpath,/tmp/tmpuri70waa/build/vendor/llama.cpp/tools/mtmd:/tmp/tmpuri70waa/build/bin: vendor/llama.cpp/common/libcommon.a vendor/llama.cpp/tools/mtmd/libmtmd.so bin/libllama.so bin/libggml.so bin/libggml-cpu.so bin/libggml-base.so && :
*** Installing project into wheel...
-- Install configuration: "Release"
-- Installing: /tmp/tmpuri70waa/wheel/platlib/lib/libggml-cpu.so
CMake Error at /tmp/tmpuri70waa/build/_deps/kleidiai_download-build/cmake_install.cmake:52 (file):
file INSTALL cannot find "/tmp/tmpuri70waa/build/bin/libkleidiai.so": No
such file or directory.
Call Stack (most recent call first):
/tmp/tmpuri70waa/build/vendor/llama.cpp/ggml/src/cmake_install.cmake:65 (include)
/tmp/tmpuri70waa/build/vendor/llama.cpp/ggml/cmake_install.cmake:47 (include)
/tmp/tmpuri70waa/build/vendor/llama.cpp/cmake_install.cmake:47 (include)
/tmp/tmpuri70waa/build/cmake_install.cmake:47 (include)



*** CMake install failed
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> llama-cpp-python

```

These CMake flags (`-DGGML_NATIVE=OFF -DGGML_CPU_ARM_ARCH=armv9-a+i8mm+dotprod -DGGML_CPU_KLEIDIAI=ON`) have already been tested on **llama.cpp**, and the build succeeds there.

The installation works for **llama-cpp-python==3.9.0**, but starting from **3.10.0**, the build fails with the error above.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.