QDQ multiply returns scale-dependent wrong values on ANE
- Dominant language
- Python
- Stars
- 152
- Forks
- 45
- Avg merge
- 1d 7m
- Merged PRs (30d)
- 12
Description
## Observed
On the Core AI → ANE-preferred path, changing the output QDQ scale makes a multiply return 2, 4 or 8 where the expected output is exactly 1. The graph has no weights or model: an all-ones FP16 input of shape `[1,32,1,64]` is split into two 16-channel branches, `a` and `b`.
```text
y = Q_out(a * Q_1/16(b))
Q_s(x) = s * clamp(round(x / s), -128, 127) # signed INT8, zero point 0
```
| Output scale | Expected | Observed | With product clamp |
|---|---:|---:|---:|
| 1/16 | 1 | 1 | 1 |
| 1/8 | 1 | 2 | 1 |
| 1/4 | 1 | 4 | 1 |
| 1/2 | 1 | 8 | 1 |
All 1,024 output elements take the value shown. There are no rounding ties or saturation. Clamping the product to `[-128 * output_scale, 127 * output_scale]` before its QDQ restores 1 in every case; the true product already lies inside that interval.
The saved graph has distinct input and output scales. The wrong values match the input branch's integer code, 16, dequantized with the output scale. This predicts the outputs but does not identify the responsible compiler or runtime stage. Each original, zero and repeat call logged one successful ANE request; zero outputs were zero and repeats matched byte for byte.
## Reproduce
The [reproducer and raw outputs](https://github.com/cadamcat/llms-on-apple-neural-engine/tree/4a8d52be5e87e1bd9db6c186b556e6fa0a1cca02/findings/coreai-qdq-multiply-scale/repro) include the exporter, saved graphs and native Swift host. With the dependencies below installed, run from that directory into a new output directory:
```sh
python verify.py
python export.py /tmp/qdq-mul-result
python run.py /tmp/qdq-mul-result
python verify.py --outputs /tmp/qdq-mul-result/host-output
```
The first command checks the recorded arrays without an Apple device. Export and execution require the Core AI SDK/runtime.
Recorded environment: Apple M5 Pro; macOS 27.0 (26A428); Xcode 27.0 (27A266a); coreai-torch 0.4.1, Torch 2.11.0, NumPy 2.3.5. This probe has not been rerun on 0.4.2.
Contributor guide
Research direction
Start in the linked repro directory by running verify.py, then export.py and run.py with the recorded Core AI environment; compare the saved graphs and raw outputs across output scales. Read the exporter and native Swift host to trace the QDQ multiply through the Core AI path. Done means identifying the responsible stage and restoring output value 1 for every listed scale without relying on product clamping.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python, pytorch, swift
- Domain
- compilers, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100