huggingface / huggingface/candle
Error: Get error "forward compatibility was attempted on non supported HW" on RTX 5090.
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
1. GPU is RTX5090. Run [mistralrs](https://github.com/EricLBuehler/mistral.rs) in Docker container. The Docker image is based on nvidia/cuda:13.1.1-cudnn-runtime-ubuntu24.04.
2. The nvidia-smi in the container is as follow:
```
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.126.09 Driver Version: 580.126.09 CUDA Version: 13.1 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 5090 Off | 00000000:01:00.0 On | N/A |
| 0% 31C P8 26W / 575W | 1487MiB / 32607MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+
```
3. Get the call stack as follow:
```
Error: DriverError(CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE, "forward compatibility was attempted on non supported HW")
0: ::bt
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/candle-core-0.9.2/src/error.rs:264:25
1: , cudarc::driver::result::DriverError> as candle_core::cuda_backend::error::WrapErr>>::w::{closure#0}
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/candle-core-0.9.2/src/cuda_backend/error.rs:60:65
2: , cudarc::driver::result::DriverError>>::map_err::, cudarc::driver::result::DriverError> as candle_core::cuda_backend::error::WrapErr>>::w::{closure#0}>
at /rustc/905b9269674ced4b5239f485609a3bf0ab02d01b/library/core/src/result.rs:968:27
3: , cudarc::driver::result::DriverError> as candle_core::cuda_backend::error::WrapErr>>::w
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/candle-core-0.9.2/src/cuda_backend/error.rs:60:14
4: ::new
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/candle-core-0.9.2/src/cuda_backend/device.rs:278:65
5: ::new_cuda
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/candle-core-0.9.2/src/device.rs:235:23
6: ::cuda_if_available
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/candle-core-0.9.2/src/device.rs:325:13
7: mistralrs_server_core::mistralrs_for_server_builder::init_device
at /mistralrs/mistralrs-server-core/src/mistralrs_for_server_builder.rs:1009:9
8: ::build_single_model::{closure#0}
at /mistralrs/mistralrs-server-core/src/mistralrs_for_server_builder.rs:624:13
9: ::build::{closure#0}
at /mistralrs/mistralrs-server-core/src/mistralrs_for_server_builder.rs:605:39
```
Refer to https://github.com/EricLBuehler/mistral.rs/issues/1858
**Does candle have any specified nvidia driver or sm version requirements?**
Thanks.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with candle-core's cuda_backend/device.rs and device.rs locations shown in the stack trace, then follow the server entry point at mistralrs-server-core/src/mistralrs_for_server_builder.rs:1009. Reproduce the reported Docker setup with the RTX 5090 and determine whether the failure is expected. Done means the supported NVIDIA driver and SM requirements are established and clearly recorded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, rust
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100