huggingface / huggingface/candle

Seeking Wisdom on Cuda

Open
#3,166 7 comments 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

Hi! I am attempting to get this working with Cuda support. Any ideas? Thank you! Works great without the CUDA flag.

**Hardware:**
- RTX480 with latest drivers
- CUDA 13.0 toolkit
- Windows
- Cudarc v17.7 (If this is a limfac, I can run a local patch etc to sync these)
- CUDA kernels known working from non-ML parts of the application with Cudarc and cuFFT bindings.

**Cargo.Toml**:
```toml

candle-core = { git = "https://github.com/huggingface/candle", package = "candle-core", features = ["cuda"] }
candle-nn = { git = "https://github.com/huggingface/candle", package = "candle-nn", features = ["cuda"] }
```

**The compile error:**
```rust
error: failed to run custom build command for `candle-kernels v0.9.2-alpha.1 (https://github.com/huggingface/candle#d4545ebb)`
note: To improve backtraces for build dependencies, set the CARGO_PROFILE_RELEASE_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation.
Caused by:
process didn't exit successfully: `C:\Users\the_a\code\Bio\daedalus\target\release\build\candle-kernels-b4b265fd7ce556ce\build-script-build` (exit code: 101)
--- stdout
cargo::rerun-if-changed=build.rs
cargo::rerun-if-changed=src/compatibility.cuh
cargo::rerun-if-changed=src/cuda_utils.cuh
cargo::rerun-if-changed=src/binary_op_macros.cuh
cargo:info=["/usr", "/usr/local/cuda", "/opt/cuda", "/usr/lib/cuda", "C:/Program Files/NVIDIA GPU Computing Toolkit", "C:/CUDA"]
cargo:rerun-if-env-changed=CUDA_COMPUTE_CAP
cargo:rustc-env=CUDA_COMPUTE_CAP=89
cargo::warning=Builder { cuda_root: None, kernel_paths: ["src\\affine.cu", "src\\binary.cu", "src\\cast.cu", "src\\conv.cu", "src\\fill.cu", "src\\indexing.cu", "src\\quantized.cu", "src\\reduce.cu", "src\\sort.cu", "src\\ternary.cu", "src\\unary.cu"], watch: [], include_paths: ["src\\binary_op_macros.cuh", "src\\compatibility.cuh", "src\\cuda_utils.cuh"], compute_cap: Some(89), out_dir: "C:\\Users\\the_a\\code\\Bio\\daedalus\\target\\release\\build\\candle-kernels-e32097457b6dd4eb\\out", extra_args: [] }
--- stderr
thread 'main' panicked at C:\Users\the_a\.cargo\registry\src\index.crates.io-1949cf8c6b5b557f\bindgen_cuda-0.1.5\src\lib.rs:306:40:
Could not find CUDA in standard locations, set it manually using Builder().set_cuda_root(...)
stack backtrace:
0: std::panicking::begin_panic_handler
at /rustc/1159e78c4747b02ef996e55082b704c09b970588/library\std\src\panicking.rs:697
1: core::panicking::panic_fmt
at /rustc/1159e78c4747b02ef996e55082b704c09b970588/library\core\src\panicking.rs:75
2: core::panicking::panic_display
at /rustc/1159e78c4747b02ef996e55082b704c09b970588/library\core\src\panicking.rs:268
3: core::option::expect_failed
at /rustc/1159e78c4747b02ef996e55082b704c09b970588/library\core\src\option.rs:2139
4: core::option::Option::expect
5: bindgen_cuda::Builder::build_ptx
6: build_script_build::main
7: std::rt::lang_start
8: std::rt::lang_start
9: std::rt::lang_start
10: std::rt::lang_start_internal::closure$0
at /rustc/1159e78c4747b02ef996e55082b704c09b970588/library\std\src\rt.rs:175
11: std::panicking::catch_unwind::do_call
at /rustc/1159e78c4747b02ef996e55082b704c09b970588/library\std\src\panicking.rs:589
12: std::panicking::catch_unwind
at /rustc/1159e78c4747b02ef996e55082b704c09b970588/library\std\src\panicking.rs:552
13: std::panic::catch_unwind
at /rustc/1159e78c4747b02ef996e55082b704c09b970588/library\std\src\panic.rs:359
14: std::rt::lang_start_internal
at /rustc/1159e78c4747b02ef996e55082b704c09b970588/library\std\src\rt.rs:171
15: std::rt::lang_start
16: main
17: invoke_main
at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
18: __scrt_common_main_seh
at D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
19: BaseThreadInitThunk
20: RtlUserThreadStart
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
```

I believe this may be due to a limitation of the `bindgen_cuda` dependency, which doesn't look in the correct location for CUDA, and doesn't check the PATH (Or LD_LIBRARY_PATH on Linux) environment variables. I have been compiling CUDA using `nvcc` in build.rs, and it is able to find CUDA without specifying any paths.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.