abetlen / abetlen/llama-cpp-python

[Request] nice-ifying build failures around hardware acceleration

未关闭
#793 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
enhancement
主要语言
Python
星标
10.6k
派生
1.4k
PR 合并指标
PR 指标待抓取

描述

Build scripts support many types of hardware acceleration: https://github.com/abetlen/llama-cpp-python/blob/v0.2.11/README.md#installation-with-hardware-acceleration

```bash
# cuBLAS
CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install llama-cpp-python
# Metal
CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python
```

If one is missing OS-level dependencies, the errors thrown can be somewhat arcane, to those not informed. The request is to nice-ify these messages, providing context and/or solutions.

For example, when installing on an AWS EC2 `g4dn.4xlarge`, without `nvcc` installed via `nvidia-cuda-toolkit`:

```none
CMake Error at /tmp/pip-build-env-xoh3dnhj/normal/lib/python3.11/site-packages/cmake/data/share/cmake-3.27/Modules/CMakeDetermineCUDACompiler.cmake:603 (message):
Failed to detect a default CUDA architecture.
```

This can be nice-ified to be:

```
Failed to detect a default CUDA architecture.
cuBLAS backend failed: do you have nvcc installed?
```

Or when accidentally pasting in the wrong `CMAKE_ARGS` (here I mixed up Metal and cuBLAS):

```none
CMake Error at vendor/llama.cpp/CMakeLists.txt:174 (find_library):
Could not find FOUNDATION_LIBRARY using the following names: Foundation
```

This can be nice-ified to be:

```
Could not find FOUNDATION_LIBRARY using the following names: Foundation
Metal backend failed: are you on a Mac?
```

In Python, `Exception`s can be re-raised using `from`: `raise Exception("Nice message.") from arcane_exc`. I am not sure if this is something supported by `CMakeLists.txt`, but I think it would be really helpful to provide better failure messages for the hardware acceleration specifics.

For context, these errors were encountered in https://github.com/ggerganov/llama.cpp/issues/3459.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。