abetlen / abetlen/llama-cpp-python

Build Failure When Enabling KleidiAI on ARMv9 in llama-cpp-python ≥ 3.10.0

Aperta
#2,095 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
10.6k
Fork
1.4k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

When I try to install the latest version of **llama-cpp-python** and enable **KleidiAI** on an **ARMv9 CPU**, I use the following command:

```bash
CMAKE_ARGS="-DGGML_NATIVE=OFF -DGGML_CPU_ARM_ARCH=armv9-a+i8mm+dotprod -DGGML_CPU_KLEIDIAI=ON" pip3 install llama-cpp-python
```

However, during the wheel-building process, it fails with the following error:

```bash
[93/94] : && /usr/bin/cmake -E rm -f vendor/llama.cpp/common/libcommon.a && /usr/bin/aarch64-linux-gnu-ar qc vendor/llama.cpp/common/libcommon.a vendor/llama.cpp/common/CMakeFiles/build_info.dir/build-info.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/arg.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/chat-parser.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/chat.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/common.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/console.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/json-partial.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/json-schema-to-grammar.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/llguidance.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/log.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/ngram-cache.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/regex-partial.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/sampling.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/speculative.cpp.o && /usr/bin/aarch64-linux-gnu-ranlib vendor/llama.cpp/common/libcommon.a && :
[94/94] : && /usr/bin/aarch64-linux-gnu-g++ -O3 -DNDEBUG vendor/llama.cpp/tools/mtmd/CMakeFiles/llama-mtmd-cli.dir/mtmd-cli.cpp.o -o vendor/llama.cpp/tools/mtmd/llama-mtmd-cli -Wl,-rpath,/tmp/tmpuri70waa/build/vendor/llama.cpp/tools/mtmd:/tmp/tmpuri70waa/build/bin: vendor/llama.cpp/common/libcommon.a vendor/llama.cpp/tools/mtmd/libmtmd.so bin/libllama.so bin/libggml.so bin/libggml-cpu.so bin/libggml-base.so && :
*** Installing project into wheel...
-- Install configuration: "Release"
-- Installing: /tmp/tmpuri70waa/wheel/platlib/lib/libggml-cpu.so
CMake Error at /tmp/tmpuri70waa/build/_deps/kleidiai_download-build/cmake_install.cmake:52 (file):
file INSTALL cannot find "/tmp/tmpuri70waa/build/bin/libkleidiai.so": No
such file or directory.
Call Stack (most recent call first):
/tmp/tmpuri70waa/build/vendor/llama.cpp/ggml/src/cmake_install.cmake:65 (include)
/tmp/tmpuri70waa/build/vendor/llama.cpp/ggml/cmake_install.cmake:47 (include)
/tmp/tmpuri70waa/build/vendor/llama.cpp/cmake_install.cmake:47 (include)
/tmp/tmpuri70waa/build/cmake_install.cmake:47 (include)



*** CMake install failed
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> llama-cpp-python

```

These CMake flags (`-DGGML_NATIVE=OFF -DGGML_CPU_ARM_ARCH=armv9-a+i8mm+dotprod -DGGML_CPU_KLEIDIAI=ON`) have already been tested on **llama.cpp**, and the build succeeds there.

The installation works for **llama-cpp-python==3.9.0**, but starting from **3.10.0**, the build fails with the error above.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.