kvcache-ai / kvcache-ai/ktransformers

kt-kernel/install.sh failed with A100

Open Beginner friendly
#2,098 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
19.5k
Forks
1.6k
Avg merge
19h 32m
Merged PRs (30d)
27

Description

### Reminder

- [x] I have read the above rules and searched the existing issues.

### System Info

Ubuntu 22.04 LTS
4xA100/80G
CUDA toolkit is 11.5, which doesn't support compute_89 (H100) or compute_90 (Blackwell).

### Reproduction

```text
cd kt-kernel
./install.sh
```

Error

```
error: subprocess-exited-with-error

× Building wheel for kt-kernel (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> No available output.

note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /usr/bin/python3 /home/swei/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmp_7tcu0ac
cwd: /home/swei/github/ktransformers/kt-kernel
Building wheel for kt-kernel (pyproject.toml) ... error
ERROR: Failed building wheel for kt-kernel
Failed to build kt-kernel
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> kt-kernel
```

### Others

Fix in install.sh: Added a detect_cuda_archs() function that checks the installed nvcc version and sets CPUINFER_CUDA_ARCHS accordingly:
- CUDA < 11.8 → 80;86 (no 89/90)
- CUDA 11.8+ → adds 89 (H100)
- CUDA 12.0+ → adds 90 (Blackwell)

```diff
diff --git a/kt-kernel/install.sh b/kt-kernel/install.sh
index 06a7d8c..e7121a0 100755
--- a/kt-kernel/install.sh
+++ b/kt-kernel/install.sh
@@ -195,6 +195,38 @@ detect_cpu_features() {
echo "$has_amx $has_avx512f $has_avx512_vnni $has_avx512_bf16 $has_avx512_vbmi"
}

+# Auto-detect supported CUDA architectures from the installed nvcc version
+detect_cuda_archs() {
+ local nvcc_path
+ nvcc_path=$(command -v nvcc 2>/dev/null || true)
+ if [ -z "$nvcc_path" ]; then
+ return 0
+ fi
+
+ # Extract nvcc major.minor version
+ local nvcc_version
+ nvcc_version=$("$nvcc_path" --version 2>/dev/null | grep -oP 'V\K[0-9]+\.[0-9]+' | head -1)
+ if [ -z "$nvcc_version" ]; then
+ return 0
+ fi
+
+ local major minor
+ major="${nvcc_version%%.*}"
+ minor="${nvcc_version#*.}"
+
+ # CUDA 11.8+ supports sm_89 (H100); CUDA 12.0+ supports sm_90 (Blackwell)
+ local archs="80;86"
+ if [ "$major" -gt 11 ] || { [ "$major" -eq 11 ] && [ "$minor" -ge 8 ]; }; then
+ archs="${archs};89"
+ fi
+ if [ "$major" -gt 12 ] || { [ "$major" -eq 12 ] && [ "$minor" -ge 0 ]; }; then
+ archs="${archs};90"
+ fi
+
+ echo "CUDA $nvcc_version detected; setting CPUINFER_CUDA_ARCHS=$archs"
+ export CPUINFER_CUDA_ARCHS="$archs"
+}
+
build_step() {
# Parse build-only flags from arguments to this function
local MANUAL_MODE=0
@@ -208,6 +240,11 @@ build_step() {
esac
done

+ # Auto-detect CUDA architectures to avoid nvcc "Unsupported gpu architecture" errors
+ if [ -z "${CPUINFER_CUDA_ARCHS:-}" ]; then
+ detect_cuda_archs
+ fi
+
# Clean local build directory to ensure a fresh CMake/configure
local REPO_ROOT
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
```

Contributor guide

Open the contributing guide

Research direction

Start with kt-kernel/install.sh and reproduce the failure with cd kt-kernel && ./install.sh on CUDA 11.5. Inspect the build step and the reported unsupported GPU-architecture error, then verify that installation completes successfully without changing a user-provided CPUINFER_CUDA_ARCHS value.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, shell
Domain
build-system
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.