huggingface / huggingface/candle

CPU matmul silently returns incorrect values for stride-zero broadcast batch

Open
#3,744 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

### Bug

On Candle 0.11.0, the baseline CPU backend silently returns incorrect values when batched `matmul` receives a stride-zero batch dimension created by `broadcast_as`. The Accelerate CPU provider returns the correct values for the same tensors.

This is related to #3253, but that report covers `broadcast_matmul` rejecting a stride-zero layout. This report concerns direct `broadcast_as(...).matmul(...)` succeeding with incorrect output.

### Minimal reproducer

```rust
use candle_core::{DType, Device, Result, Tensor};

fn main() -> Result<()> {
let device = Device::Cpu;
let left = Tensor::ones((1, 32, 32), DType::F32, &device)?;
let right = Tensor::ones((32, 32, 32), DType::F32, &device)?;

let stride_zero = left.broadcast_as((32, 32, 32))?.matmul(&right)?;
let eager = left.repeat((32, 1, 1))?.matmul(&right)?;

assert_eq!(
stride_zero.flatten_all()?.to_vec1::()?,
eager.flatten_all()?.to_vec1::()?,
);
Ok(())
}
```

With baseline `candle-core = "0.11.0"`, the assertion fails. With `candle-core = { version = "0.11.0", features = ["accelerate"] }` on the same Apple Silicon host, it passes.

### Expected behavior

`matmul` should either honor the stride-zero batch layout and match the eagerly expanded input, or reject the unsupported layout. It should not succeed with incorrect values.

### Environment

- Candle 0.11.0
- Rust 1.94.1, edition 2024
- macOS arm64 / Apple M4 Max
- Baseline CPU provider: incorrect values
- Accelerate CPU provider: correct values

I can prepare a PR once the desired baseline CPU behavior and regression-test location are confirmed.

Contributor guide

No contributing guide indexed for this repository

Research direction

Run the minimal reproducer and compare direct broadcast_as(...).matmul(...) with the eagerly repeated input on the baseline CPU provider. Trace the matmul entry point into the baseline CPU implementation, then add a regression test once its location is identified; done means the stride-zero layout matches the eager result or is rejected rather than producing incorrect values.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.