huggingface / huggingface/candle

Compilation errors due to incorrect GPU architecture

Open
#2,610 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

I got these errors:
src/quantized.cu(1962): error: identifier "__dp4a" is undefined
sumi = __dp4a(v[i], u[i], sumi);
^

18 errors detected in the compilation of "src/quantized.cu".
thread 'main' panicked at /home/remote/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen_cuda-0.1.5/src/lib.rs:391:13:
nvcc error while compiling "src/quantized.cu":

# CLI "nvcc" "--gpu-architecture=sm_60" "--ptx" "--default-stream" "per-thread" "--output-directory" "/projects/ai/candle/candle/candle-kernels/target/debug/build/candle-kernels-2b82da83d6e51215/out" "-Isrc" "-I/usr/include" "src/quantized.cu"

They were solved after some ChatGPT and putting this:

export CUDA_COMPUTE_CAP=61

However it would be better to solve this in a more generic way.

Just as an FYI. This happens with Ubuntu 24.04, CUDA 12.6 and 3 Tesla P100.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with candle-kernels/src/quantized.cu and the build invocation showing nvcc --gpu-architecture=sm_60. Check how CUDA_COMPUTE_CAP is read and how the GPU architecture is selected, using the reported Tesla P100 failure as the reproduction case. Done means the kernels compile with the appropriate architecture without requiring a manually exported environment variable.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
build-system, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.