huggingface / huggingface/candle
Tracking: Metal issues with examples
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
When compiled with the `--features metal` flag some of the examples don't run.
- `based` -> `cargo run --example based --features metal --release -- --prompt "Flying monkeys are" --which 1b-50b --sample-len 100`
- `Error: Metal error Invalid matmul arguments [144, 9, 3, 1] [5376, 112, 1792, 1] (3, 112, 3)`
- `beit` -> `cargo run --example beit --release --features metal -- --image candle-examples/examples/yolo-v8/assets/bike.jpg`
- `Error: Metal strided affine I64 not implemented`
- `chatglm` -> `cargo run --example chatglm --release --features metal -- --prompt "部署门槛较低等众多优秀特 "`
- `Error: Metal error Invalid matmul arguments [128, 4096, 1] [128, 1, 4096] (10, 10, 128)`
- `custom-ops` -> `cargo run --example custom-ops --features metal`
- `Error: Metal error no metal implementation for layer-norm` (Need to add metal kernel for this to work)
- `eva2` -> `cargo run --example eva2 --release --features metal -- --image candle-examples/examples/yolo-v8/assets/bike.jpg`
- `Error: Metal error Invalid matmul arguments [787200, 65600, 64, 1] [64, 64, 1, 768] (1025, 1025, 64)`
- `silero-vad` -> `rec -t raw -r 48000 -b 16 -c 1 -e signed-integer - trim 0 5 | sox -t raw -r 48000 -b 16 -c 1 -e signed-integer - -t raw -r 16000 -b 16 -c 1 -e signed-integer - | cargo run --example silero-vad --release --features onnx,metal -- --sample-rate 16000`
- `candle-examples/examples/silero-vad/main.rs:180:60: called `Result::unwrap()` on an `Err` value: device mismatch in cmp, lhs: Metal { gpu_id: 4294968703 }, rhs: Cpu`
- `stable-diffusion` -> `cargo run --example stable-diffusion --release --features=metal -- --prompt "a cosmonaut on a horse (hd, realistic, high-def)"`
- `Error: Metal error Metal seed must be less than or equal to u32::MAX`
System Info:
cargo 1.85.1 (d73d2caf9 2024-12-31)
rustc 1.85.1 (4eb161250 2025-03-15)
Apple M3 Max (36 GB)
MacOS 15.3.2
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the listed commands for the affected examples, starting with the Metal failures and the device-mismatch location at candle-examples/examples/silero-vad/main.rs:180. Inspect the relevant Metal backend paths and example entry points for each reported error. Done means the listed examples run successfully with their stated Metal feature commands on the reported macOS system.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos, rust
- Domain
- machine-learning, operating-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100