kvcache-ai / kvcache-ai/ktransformers
kt-kernel/install.sh failed with A100
- Dominant language
- Python
- Stars
- 19.5k
- Forks
- 1.6k
- Avg merge
- 19h 32m
- Merged PRs (30d)
- 27
Description
### Reminder
- [x] I have read the above rules and searched the existing issues.
### System Info
Ubuntu 22.04 LTS
4xA100/80G
CUDA toolkit is 11.5, which doesn't support compute_89 (H100) or compute_90 (Blackwell).
### Reproduction
```text
cd kt-kernel
./install.sh
```
Error
```
error: subprocess-exited-with-error
× Building wheel for kt-kernel (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> No available output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /usr/bin/python3 /home/swei/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmp_7tcu0ac
cwd: /home/swei/github/ktransformers/kt-kernel
Building wheel for kt-kernel (pyproject.toml) ... error
ERROR: Failed building wheel for kt-kernel
Failed to build kt-kernel
error: failed-wheel-build-for-install
× Failed to build installable wheels for some pyproject.toml based projects
╰─> kt-kernel
```
### Others
Fix in install.sh: Added a detect_cuda_archs() function that checks the installed nvcc version and sets CPUINFER_CUDA_ARCHS accordingly:
- CUDA < 11.8 → 80;86 (no 89/90)
- CUDA 11.8+ → adds 89 (H100)
- CUDA 12.0+ → adds 90 (Blackwell)
```diff
diff --git a/kt-kernel/install.sh b/kt-kernel/install.sh
index 06a7d8c..e7121a0 100755
--- a/kt-kernel/install.sh
+++ b/kt-kernel/install.sh
@@ -195,6 +195,38 @@ detect_cpu_features() {
echo "$has_amx $has_avx512f $has_avx512_vnni $has_avx512_bf16 $has_avx512_vbmi"
}
+# Auto-detect supported CUDA architectures from the installed nvcc version
+detect_cuda_archs() {
+ local nvcc_path
+ nvcc_path=$(command -v nvcc 2>/dev/null || true)
+ if [ -z "$nvcc_path" ]; then
+ return 0
+ fi
+
+ # Extract nvcc major.minor version
+ local nvcc_version
+ nvcc_version=$("$nvcc_path" --version 2>/dev/null | grep -oP 'V\K[0-9]+\.[0-9]+' | head -1)
+ if [ -z "$nvcc_version" ]; then
+ return 0
+ fi
+
+ local major minor
+ major="${nvcc_version%%.*}"
+ minor="${nvcc_version#*.}"
+
+ # CUDA 11.8+ supports sm_89 (H100); CUDA 12.0+ supports sm_90 (Blackwell)
+ local archs="80;86"
+ if [ "$major" -gt 11 ] || { [ "$major" -eq 11 ] && [ "$minor" -ge 8 ]; }; then
+ archs="${archs};89"
+ fi
+ if [ "$major" -gt 12 ] || { [ "$major" -eq 12 ] && [ "$minor" -ge 0 ]; }; then
+ archs="${archs};90"
+ fi
+
+ echo "CUDA $nvcc_version detected; setting CPUINFER_CUDA_ARCHS=$archs"
+ export CPUINFER_CUDA_ARCHS="$archs"
+}
+
build_step() {
# Parse build-only flags from arguments to this function
local MANUAL_MODE=0
@@ -208,6 +240,11 @@ build_step() {
esac
done
+ # Auto-detect CUDA architectures to avoid nvcc "Unsupported gpu architecture" errors
+ if [ -z "${CPUINFER_CUDA_ARCHS:-}" ]; then
+ detect_cuda_archs
+ fi
+
# Clean local build directory to ensure a fresh CMake/configure
local REPO_ROOT
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
```
Contributor guide
Research direction
Start with kt-kernel/install.sh and reproduce the failure with cd kt-kernel && ./install.sh on CUDA 11.5. Inspect the build step and the reported unsupported GPU-architecture error, then verify that installation completes successfully without changing a user-provided CPUINFER_CUDA_ARCHS value.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, shell
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100