huggingface / huggingface/candle
Cuda memory requirements for stable-diffusion example
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
I believe the answer is no but would like to confirm.
Can this example run with the cuda and or cudnn features enabled?
```
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.86.10 Driver Version: 535.86.10 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA RTX A4500 On | 00000000:06:00.0 On | Off |
| 30% 52C P0 68W / 200W | 1382MiB / 20470MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 4887 G /usr/lib/xorg/Xorg 698MiB |
| 0 N/A N/A 5220 G /usr/bin/gnome-shell 139MiB |
| 0 N/A N/A 23177 G ...ures=SpareRendererForSitePerProcess 38MiB |
| 0 N/A N/A 23216 G ...,WinRetrieveSuggestionsOnlyOnDemand 47MiB |
| 0 N/A N/A 23384 G ...55655952,1848653116653008658,262144 188MiB |
| 0 N/A N/A 27120 G ...ures=SpareRendererForSitePerProcess 44MiB |
+---------------------------------------------------------------------------------------+
```
```
cargo run --example stable-diffusion --release --features cuda --features cudnn -- --prompt "a rusty robot holding a fire torch"
warning: some crates are on edition 2021 which defaults to `resolver = "2"`, but virtual workspaces default to `resolver = "1"`
note: to keep the current resolver, specify `workspace.resolver = "1"` in the workspace root's manifest
note: to use the edition 2021 resolver, specify `workspace.resolver = "2"` in the workspace root's manifest
Compiling candle-examples v0.1.3 (/mnt/source1/djbGR/ruststuffs/candle/candle-examples)
Finished release [optimized] target(s) in 7.29s
Running `target/release/examples/stable-diffusion --prompt 'a rusty robot holding a fire torch'`
tokenizer.json [00:00:00] [███████████████████████████████████████████████████████████████████████████████████████████████████████████] 2.12 MiB/2.12 MiB 7.97 MiB/s (0s)Running with prompt "a rusty robot holding a fire torch".
Building the Clip transformer.
text_encoder/model.safetensors [00:00:11] [█████████████████████████████████████████████████████████████████████████████████████████] 1.27 GiB/1.27 GiB 110.75 MiB/s (0s)Building the autoencoder.
..sion_pytorch_model.safetensors [00:00:03] [████████████████████████████████████████████████████████████████████████████████████] 319.14 MiB/319.14 MiB 99.32 MiB/s (0s)Building the unet.
..sion_pytorch_model.safetensors [00:00:30] [███████████████████████████████████████████████████████████████████████████████████████] 3.23 GiB/3.23 GiB 109.06 MiB/s (0s)starting sampling
Error: DriverError(CUDA_ERROR_OUT_OF_MEMORY, "out of memory")
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.