huggingface / huggingface/candle

assertion failed: !ptr.is_null() with Stable Diffusion img2img on metal

Open
#2,417 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

Command:
```
cargo run --example stable-diffusion --features=metal -- --prompt "a cosmonaut on a horse (hd, realistic, high-def)" --img2img [path_to_img]
```

Stacktrace
```
thread 'main' panicked at /Users/worm/.cargo/registry/src/index.crates.io-6f17d22bba15001f/foreign-types-shared-0.3.1/src/lib.rs:72:9:
assertion failed: !ptr.is_null()
stack backtrace:
0: rust_begin_unwind
at /rustc/3f5fd8dd41153bc5fdca9427e9e05be2c767ba23/library/std/src/panicking.rs:652:5
1: core::panicking::panic_fmt
at /rustc/3f5fd8dd41153bc5fdca9427e9e05be2c767ba23/library/core/src/panicking.rs:72:14
2: core::panicking::panic
at /rustc/3f5fd8dd41153bc5fdca9427e9e05be2c767ba23/library/core/src/panicking.rs:146:5
3: foreign_types_shared::ForeignTypeRef::from_ptr
at /Users/worm/.cargo/registry/src/index.crates.io-6f17d22bba15001f/foreign-types-shared-0.3.1/src/lib.rs:72:9
4: ::deref
at /Users/worm/.cargo/registry/src/index.crates.io-6f17d22bba15001f/metal-0.27.0/src/lib.rs:169:9
5: <&metal::buffer::Buffer as candle_metal_kernels::utils::EncoderParam>::set_param
at ./candle-metal-kernels/src/utils.rs:125:43
6: candle_metal_kernels::utils::set_param
at ./candle-metal-kernels/src/utils.rs:69:5
7: candle_metal_kernels::call_im2col_strided
at ./candle-metal-kernels/src/utils.rs:158:13
8: ::conv2d
at ./candle-core/src/metal_backend/mod.rs:962:9
9: candle_core::storage::Storage::conv2d
at ./candle-core/src/storage.rs:443:25
10: candle_core::conv::::conv2d_single_group
at ./candle-core/src/conv.rs:258:13
11: candle_core::conv::::conv2d
at ./candle-core/src/conv.rs:301:13
12: ::forward
at ./candle-nn/src/conv.rs:214:17
13: ::forward
at ./candle-transformers/src/models/with_tracing.rs:85:9
14: candle_transformers::models::stable_diffusion::resnet::ResnetBlock2D::forward
at ./candle-transformers/src/models/stable_diffusion/resnet.rs:124:18
15: ::forward
at ./candle-transformers/src/models/stable_diffusion/unet_2d_blocks.rs:180:18
16: candle_core::tensor::Tensor::apply
at ./candle-core/src/tensor.rs:2285:9
17: candle_transformers::models::stable_diffusion::vae::Encoder::forward
at ./candle-transformers/src/models/stable_diffusion/vae.rs:138:18
18: candle_transformers::models::stable_diffusion::vae::AutoEncoderKL::encode
at ./candle-transformers/src/models/stable_diffusion/vae.rs:375:18
19: stable_diffusion::run
at ./candle-examples/examples/stable-diffusion/main.rs:521:18
20: stable_diffusion::main
at ./candle-examples/examples/stable-diffusion/main.rs:630:5
21: core::ops::function::FnOnce::call_once
at /rustc/3f5fd8dd41153bc5fdca9427e9e05be2c767ba23/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the command with the metal feature and the img2img option, then inspect candle-metal-kernels/src/utils.rs and candle-core/src/metal_backend/mod.rs at the stack-trace locations. Trace the call from candle-examples/examples/stable-diffusion/main.rs through the VAE encoder and convolution path. Done means the command no longer panics and completes img2img generation on Metal.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.