huggingface / huggingface/candle
assertion failed: !ptr.is_null() with Stable Diffusion img2img on metal
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
Command:
```
cargo run --example stable-diffusion --features=metal -- --prompt "a cosmonaut on a horse (hd, realistic, high-def)" --img2img [path_to_img]
```
Stacktrace
```
thread 'main' panicked at /Users/worm/.cargo/registry/src/index.crates.io-6f17d22bba15001f/foreign-types-shared-0.3.1/src/lib.rs:72:9:
assertion failed: !ptr.is_null()
stack backtrace:
0: rust_begin_unwind
at /rustc/3f5fd8dd41153bc5fdca9427e9e05be2c767ba23/library/std/src/panicking.rs:652:5
1: core::panicking::panic_fmt
at /rustc/3f5fd8dd41153bc5fdca9427e9e05be2c767ba23/library/core/src/panicking.rs:72:14
2: core::panicking::panic
at /rustc/3f5fd8dd41153bc5fdca9427e9e05be2c767ba23/library/core/src/panicking.rs:146:5
3: foreign_types_shared::ForeignTypeRef::from_ptr
at /Users/worm/.cargo/registry/src/index.crates.io-6f17d22bba15001f/foreign-types-shared-0.3.1/src/lib.rs:72:9
4: ::deref
at /Users/worm/.cargo/registry/src/index.crates.io-6f17d22bba15001f/metal-0.27.0/src/lib.rs:169:9
5: <&metal::buffer::Buffer as candle_metal_kernels::utils::EncoderParam>::set_param
at ./candle-metal-kernels/src/utils.rs:125:43
6: candle_metal_kernels::utils::set_param
at ./candle-metal-kernels/src/utils.rs:69:5
7: candle_metal_kernels::call_im2col_strided
at ./candle-metal-kernels/src/utils.rs:158:13
8: ::conv2d
at ./candle-core/src/metal_backend/mod.rs:962:9
9: candle_core::storage::Storage::conv2d
at ./candle-core/src/storage.rs:443:25
10: candle_core::conv::::conv2d_single_group
at ./candle-core/src/conv.rs:258:13
11: candle_core::conv::::conv2d
at ./candle-core/src/conv.rs:301:13
12: ::forward
at ./candle-nn/src/conv.rs:214:17
13: ::forward
at ./candle-transformers/src/models/with_tracing.rs:85:9
14: candle_transformers::models::stable_diffusion::resnet::ResnetBlock2D::forward
at ./candle-transformers/src/models/stable_diffusion/resnet.rs:124:18
15: ::forward
at ./candle-transformers/src/models/stable_diffusion/unet_2d_blocks.rs:180:18
16: candle_core::tensor::Tensor::apply
at ./candle-core/src/tensor.rs:2285:9
17: candle_transformers::models::stable_diffusion::vae::Encoder::forward
at ./candle-transformers/src/models/stable_diffusion/vae.rs:138:18
18: candle_transformers::models::stable_diffusion::vae::AutoEncoderKL::encode
at ./candle-transformers/src/models/stable_diffusion/vae.rs:375:18
19: stable_diffusion::run
at ./candle-examples/examples/stable-diffusion/main.rs:521:18
20: stable_diffusion::main
at ./candle-examples/examples/stable-diffusion/main.rs:630:5
21: core::ops::function::FnOnce::call_once
at /rustc/3f5fd8dd41153bc5fdca9427e9e05be2c767ba23/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the command with the metal feature and the img2img option, then inspect candle-metal-kernels/src/utils.rs and candle-core/src/metal_backend/mod.rs at the stack-trace locations. Trace the call from candle-examples/examples/stable-diffusion/main.rs through the VAE encoder and convolution path. Done means the command no longer panics and completes img2img generation on Metal.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100