abetlen / abetlen/llama-cpp-python

Build Failure When Enabling KleidiAI on ARMv9 in llama-cpp-python ≥ 3.10.0

オープン
#2,095 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Python
スター
10.6k
フォーク
1.4k
PR マージ指標
PR 指標を取得中

説明

When I try to install the latest version of **llama-cpp-python** and enable **KleidiAI** on an **ARMv9 CPU**, I use the following command:

```bash
CMAKE_ARGS="-DGGML_NATIVE=OFF -DGGML_CPU_ARM_ARCH=armv9-a+i8mm+dotprod -DGGML_CPU_KLEIDIAI=ON" pip3 install llama-cpp-python
```

However, during the wheel-building process, it fails with the following error:

```bash
[93/94] : && /usr/bin/cmake -E rm -f vendor/llama.cpp/common/libcommon.a && /usr/bin/aarch64-linux-gnu-ar qc vendor/llama.cpp/common/libcommon.a vendor/llama.cpp/common/CMakeFiles/build_info.dir/build-info.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/arg.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/chat-parser.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/chat.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/common.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/console.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/json-partial.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/json-schema-to-grammar.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/llguidance.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/log.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/ngram-cache.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/regex-partial.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/sampling.cpp.o vendor/llama.cpp/common/CMakeFiles/common.dir/speculative.cpp.o && /usr/bin/aarch64-linux-gnu-ranlib vendor/llama.cpp/common/libcommon.a && :
[94/94] : && /usr/bin/aarch64-linux-gnu-g++ -O3 -DNDEBUG vendor/llama.cpp/tools/mtmd/CMakeFiles/llama-mtmd-cli.dir/mtmd-cli.cpp.o -o vendor/llama.cpp/tools/mtmd/llama-mtmd-cli -Wl,-rpath,/tmp/tmpuri70waa/build/vendor/llama.cpp/tools/mtmd:/tmp/tmpuri70waa/build/bin: vendor/llama.cpp/common/libcommon.a vendor/llama.cpp/tools/mtmd/libmtmd.so bin/libllama.so bin/libggml.so bin/libggml-cpu.so bin/libggml-base.so && :
*** Installing project into wheel...
-- Install configuration: "Release"
-- Installing: /tmp/tmpuri70waa/wheel/platlib/lib/libggml-cpu.so
CMake Error at /tmp/tmpuri70waa/build/_deps/kleidiai_download-build/cmake_install.cmake:52 (file):
file INSTALL cannot find "/tmp/tmpuri70waa/build/bin/libkleidiai.so": No
such file or directory.
Call Stack (most recent call first):
/tmp/tmpuri70waa/build/vendor/llama.cpp/ggml/src/cmake_install.cmake:65 (include)
/tmp/tmpuri70waa/build/vendor/llama.cpp/ggml/cmake_install.cmake:47 (include)
/tmp/tmpuri70waa/build/vendor/llama.cpp/cmake_install.cmake:47 (include)
/tmp/tmpuri70waa/build/cmake_install.cmake:47 (include)



*** CMake install failed
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> llama-cpp-python

```

These CMake flags (`-DGGML_NATIVE=OFF -DGGML_CPU_ARM_ARCH=armv9-a+i8mm+dotprod -DGGML_CPU_KLEIDIAI=ON`) have already been tested on **llama.cpp**, and the build succeeds there.

The installation works for **llama-cpp-python==3.9.0**, but starting from **3.10.0**, the build fails with the error above.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。