huggingface / huggingface/candle

Tracking: Metal issues with examples

Open
#2,832 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

When compiled with the `--features metal` flag some of the examples don't run.

- `based` -> `cargo run --example based --features metal --release -- --prompt "Flying monkeys are" --which 1b-50b --sample-len 100`
- `Error: Metal error Invalid matmul arguments [144, 9, 3, 1] [5376, 112, 1792, 1] (3, 112, 3)`
- `beit` -> `cargo run --example beit --release --features metal -- --image candle-examples/examples/yolo-v8/assets/bike.jpg`
- `Error: Metal strided affine I64 not implemented`
- `chatglm` -> `cargo run --example chatglm --release --features metal -- --prompt "部署门槛较低等众多优秀特 "`
- `Error: Metal error Invalid matmul arguments [128, 4096, 1] [128, 1, 4096] (10, 10, 128)`
- `custom-ops` -> `cargo run --example custom-ops --features metal`
- `Error: Metal error no metal implementation for layer-norm` (Need to add metal kernel for this to work)
- `eva2` -> `cargo run --example eva2 --release --features metal -- --image candle-examples/examples/yolo-v8/assets/bike.jpg`
- `Error: Metal error Invalid matmul arguments [787200, 65600, 64, 1] [64, 64, 1, 768] (1025, 1025, 64)`
- `silero-vad` -> `rec -t raw -r 48000 -b 16 -c 1 -e signed-integer - trim 0 5 | sox -t raw -r 48000 -b 16 -c 1 -e signed-integer - -t raw -r 16000 -b 16 -c 1 -e signed-integer - | cargo run --example silero-vad --release --features onnx,metal -- --sample-rate 16000`
- `candle-examples/examples/silero-vad/main.rs:180:60: called `Result::unwrap()` on an `Err` value: device mismatch in cmp, lhs: Metal { gpu_id: 4294968703 }, rhs: Cpu`
- `stable-diffusion` -> `cargo run --example stable-diffusion --release --features=metal -- --prompt "a cosmonaut on a horse (hd, realistic, high-def)"`
- `Error: Metal error Metal seed must be less than or equal to u32::MAX`

System Info:
cargo 1.85.1 (d73d2caf9 2024-12-31)
rustc 1.85.1 (4eb161250 2025-03-15)
Apple M3 Max (36 GB)
MacOS 15.3.2

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the listed commands for the affected examples, starting with the Metal failures and the device-mismatch location at candle-examples/examples/silero-vad/main.rs:180. Inspect the relevant Metal backend paths and example entry points for each reported error. Done means the listed examples run successfully with their stated Metal feature commands on the reported macOS system.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, rust
Domain
machine-learning, operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.