huggingface / huggingface/candle

Error: Get error "forward compatibility was attempted on non supported HW" on RTX 5090.

Open
#3,358 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

1. GPU is RTX5090. Run [mistralrs](https://github.com/EricLBuehler/mistral.rs) in Docker container. The Docker image is based on nvidia/cuda:13.1.1-cudnn-runtime-ubuntu24.04.

2. The nvidia-smi in the container is as follow:

```
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.126.09 Driver Version: 580.126.09 CUDA Version: 13.1 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 5090 Off | 00000000:01:00.0 On | N/A |
| 0% 31C P8 26W / 575W | 1487MiB / 32607MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+
```
3. Get the call stack as follow:
```
Error: DriverError(CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE, "forward compatibility was attempted on non supported HW")
0: ::bt
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/candle-core-0.9.2/src/error.rs:264:25
1: , cudarc::driver::result::DriverError> as candle_core::cuda_backend::error::WrapErr>>::w::{closure#0}
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/candle-core-0.9.2/src/cuda_backend/error.rs:60:65
2: , cudarc::driver::result::DriverError>>::map_err::, cudarc::driver::result::DriverError> as candle_core::cuda_backend::error::WrapErr>>::w::{closure#0}>
at /rustc/905b9269674ced4b5239f485609a3bf0ab02d01b/library/core/src/result.rs:968:27
3: , cudarc::driver::result::DriverError> as candle_core::cuda_backend::error::WrapErr>>::w
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/candle-core-0.9.2/src/cuda_backend/error.rs:60:14
4: ::new
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/candle-core-0.9.2/src/cuda_backend/device.rs:278:65
5: ::new_cuda
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/candle-core-0.9.2/src/device.rs:235:23
6: ::cuda_if_available
at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/candle-core-0.9.2/src/device.rs:325:13
7: mistralrs_server_core::mistralrs_for_server_builder::init_device
at /mistralrs/mistralrs-server-core/src/mistralrs_for_server_builder.rs:1009:9
8: ::build_single_model::{closure#0}
at /mistralrs/mistralrs-server-core/src/mistralrs_for_server_builder.rs:624:13
9: ::build::{closure#0}
at /mistralrs/mistralrs-server-core/src/mistralrs_for_server_builder.rs:605:39
```

Refer to https://github.com/EricLBuehler/mistral.rs/issues/1858

**Does candle have any specified nvidia driver or sm version requirements?**

Thanks.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with candle-core's cuda_backend/device.rs and device.rs locations shown in the stack trace, then follow the server entry point at mistralrs-server-core/src/mistralrs_for_server_builder.rs:1009. Reproduce the reported Docker setup with the RTX 5090 and determine whether the failure is expected. Done means the supported NVIDIA driver and SM requirements are established and clearly recorded.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, rust
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.