# [Bug] QDQ optimization changes numeric results: fused integer-domain ops (QLinearAdd/QLinearConv/QLinearAveragePool) off by one or more quantization steps (diff == k * scale)

Nobody has claimed this yet.

[518_minimal.onnx.zip](https://github.com/user-attachments/files/31644336/518_minimal.onnx.zip)

**Describe the bug**

In quantized (QDQ) models, level-2 graph optimizations (`ORT_ENABLE_EXTENDED` / `ORT_ENABLE_ALL`, the default) can produce a graph output that differs from the unoptimized run by an integer number of quantization steps (`diff == k * scale`, k = 1 or 2).

At level 2, `QDQSelectorActionTransformer` fuses quantized op chains into integer-domain ops (`Add -> QuantizeLinear` into `QLinearAdd`, `Conv -> QDQ` into `QLinearConv`, `AveragePool -> QDQ` into `QLinearAveragePool`), while `NhwcTransformer` converts convs to NHWC layout. The integer-domain result of the fused ops differs from the float op + `QuantizeLinear` path by one LSB, and the discrepancy surfaces at `DequantizeLinear` graph outputs.

The bug is not specific to shared-`QuantizeLinear` topologies: it also occurs in plain single-chain QDQ `Conv` / `AveragePool` models (see Additional notes), so the root cause is in the fused integer-domain op itself, not in tensor sharing.

`ORT_DISABLE_ALL` and `ORT_ENABLE_BASIC` are bit-identical to each other; the bug first appears at level 2 (`ORT_ENABLE_EXTENDED`).

**Urgency**

none

**System information**

- OS Platform and Distribution: Linux Ubuntu 20.04.4 LTS (x86_64), kernel 5.15.0-70-generic
- ONNX Runtime installed from (source or binary): binary (pip, CPU package)
- ONNX Runtime version: 1.28.0 (also reproduced on 1.23.2)
- Python version: 3.11.7
- ONNX version: 1.22.0
- numpy version: 1.26.4

**To Reproduce**

**1. Synthetic minimal model (17 nodes, synthetic weights, deterministic trigger — no attachment needed):**

```python
import numpy as np
import onnx
import onnxruntime as ort
from onnx import helper, TensorProto, numpy_helper

# trigger values (shared QuantizeLinear scale = sa2 = 0.38; zero_point = 128)
sx, sw1, s1, sa1, sw2, s2, sa2 = 0.09, 0.002, 0.06, 0.37, 0.01, 0.15, 0.38
ZP = 128

rng = np.random.default_rng(0)
W1 = np.clip(np.round((rng.random((4, 4, 3, 3)) * 2 - 1).astype(np.float32) * 5.0 / sw1), 0, 255).astype(np.uint8)
W2 = np.clip(np.round((rng.random((4, 4, 3, 3)) * 2 - 1).astype(np.float32) * 5.0 / sw2), 0, 255).astype(np.uint8)

x = helper.make_tensor_value_info("x", TensorProto.FLOAT, [1, 4, 8, 8])
y = helper.make_tensor_value_info("output", TensorProto.FLOAT, [1, 4, 8, 8])

def qdq(name):
q = helper.make_node("QuantizeLinear", [name, f"s_{name}", f"zp_{name}"], [f"q_{name}"])
d = helper.make_node("DequantizeLinear", [f"q_{name}", f"s_{name}", f"zp_{name}"], [f"dq_{name}"])
return q, d

def S(name, v): return numpy_helper.from_array(np.array(v, dtype=np.float32).reshape(()), f"s_{name}")
def Z(name): return numpy_helper.from_array(np.array(ZP, dtype=np.uint8).reshape(()), f"zp_{name}")

nodes = []
qx, dx = qdq("x"); nodes += [qx, dx]
nodes.append(helper.make_node("DequantizeLinear", ["w1", "s_w1", "zp_w1"], ["w1_dq"]))
nodes.append(helper.make_node("Conv", ["dq_x", "w1_dq"], ["c1"], pads=[1, 1, 1, 1]))
qc1, dc1 = qdq("c1"); nodes += [qc1, dc1]
nodes.append(helper.make_node("Add", ["dq_c1", "dq_x"], ["add1"]))
qa1, _ = qdq("add1"); nodes.append(qa1)
# shared Q: q_add1 feeds TWO DequantizeLinear consumers (residual skip + next layer)
nodes.append(helper.make_node("DequantizeLinear", ["q_add1", "s_add1", "zp_add1"], ["skip1"]))
nodes.append(helper.make_node("DequantizeLinear", ["q_add1", "s_add1", "zp_add1"], ["next1"]))
nodes.append(helper.make_node("DequantizeLinear", ["w2", "s_w2", "zp_w2"], ["w2_dq"]))
nodes.append(helper.make_node("Conv", ["next1", "w2_dq"], ["c2"], pads=[1, 1, 1, 1]))
qc2, dc2 = qdq("c2"); nodes += [qc2, dc2]
nodes.append(helper.make_node("Add", ["dq_c2", "skip1"], ["add2"]))
qa2, da2 = qdq("add2"); nodes += [qa2, da2]
da2.output[0] = "output"

inits = [numpy_helper.from_array(W1, "w1"), numpy_helper.from_array(W2, "w2"),
S("x", sx), Z("x"), S("w1", sw1), Z("w1"), S("c1", s1), Z("c1"),
S("add1", sa1), Z("add1"), S("w2", sw2), Z("w2"), S("c2", s2), Z("c2"),
S("add2", sa2), Z("add2")]
model = helper.make_model(helper.make_graph(nodes, "qdq_res", [x], [y], inits),
opset_imports=[helper.make_opsetid("", 13)])
model.ir_version = 9

feed = {"x": np.full([1, 4, 8, 8], 1.0, dtype=np.float32)}
def run(lvl):
so = ort.SessionOptions(); so.graph_optimization_level = lvl; so.log_severity_level = 3
return ort.InferenceSession(model.SerializeToString(), so).run(None, feed)[0]

off = run(ort.GraphOptimizationLevel.ORT_DISABLE_ALL)
on = run(ort.GraphOptimizationLevel.ORT_ENABLE_ALL)
d = np.abs(off.astype(np.float64) - on.astype(np.float64))
print("max diff =", d.max(), " #elements =", int(np.sum(d > 1e-6)))
# max diff = 0.3800001 (= the shared QuantizeLinear scale sa2) #elements = 25
```

**2. Real-world model (`518_minimal.onnx`, attached)** — a 27-node graph distilled from a 47-node MobileNetEdgeTPU QDQ subgraph carrying the actual trained weights. Its shared `QuantizeLinear` scale is 0.3718. Feed the model's two graph inputs with `randn * 2`, seed 7:

```python
import numpy as np, onnxruntime as ort
MODEL = "518_minimal.onnx"
feeds = {}
for inp in ort.InferenceSession(MODEL).get_inputs():
feeds[inp.name] = (np.random.RandomState(7).randn(*[d if d > 0 else 1 for d in inp.shape]) * 2).astype(np.float32)
so_off = ort.SessionOptions(); so_off.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL
off = ort.InferenceSession(MODEL, so_off).run(None, feeds)
so_on = ort.SessionOptions(); so_on.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
on = ort.InferenceSession(MODEL, so_on).run(None, feeds)
for o, a in zip(off, on):
d = np.abs(o.astype(np.float64) - a.astype(np.float64))
print("max diff =", d.max(), " #elements =", int(np.sum(d > 1e-6)))
# max diff = 0.371778 #elements = 93 (input-dependent: seed 3 gives 2 steps, max diff 0.7436, 36 elements; some seeds show no diff)
```

**Expected behavior**

Graph optimizations are semantics-preserving: the optimized output should be bit-identical to the unoptimized `DequantizeLinear` path.

**Actual behavior**

At `ORT_ENABLE_EXTENDED` / `ORT_ENABLE_ALL`, output elements are off by one or two uint8 quantization steps. For the synthetic reproducer: 25 elements off by 0.38. For 518_minimal.onnx: up to 93 elements off by 0.3718 (1 step), and 0.7436 (2 steps) on other inputs. `ORT_DISABLE_ALL` / `ORT_ENABLE_BASIC` are bit-identical.

**Additional notes**

- The same off-by-one occurs without any shared `QuantizeLinear`: single-chain QDQ models 3168.onnx (`QLinearConv`) and 3193.onnx (`QLinearAveragePool`) show max diff 0.0035 (one step) on all 8 seeds tested (10–22 and 155–164 elements respectively); 6572.onnx (47 nodes, same 0.3718-scale structure as 518) shows up to 2 steps on the shared-scale output. These files are in the `crash_models/` directory alongside this issue.
- The optimized graph keeps the `DequantizeLinear` graph outputs as-is; the LSB error originates in the fused integer-domain ops (`QLinearAdd` / `QLinearConv` / `QLinearAveragePool`, NHWC layout with `Transpose` / `ReorderOutput` around them).
- Reproduced by comparing ORT with optimizations disabled (`ORT_DISABLE_ALL`) vs enabled (`ORT_ENABLE_ALL`).

Reproduce the synthetic model and compare ORT_DISABLE_ALL with ORT_ENABLE_ALL, then inspect the QDQSelectorActionTransformer and NhwcTransformer paths named in the report. Compare the fused QLinearAdd, QLinearConv, and QLinearAveragePool results with the unfused QDQ path, using the listed crash_models files for additional cases. Done means optimized and unoptimized outputs are bit-identical.

Written by the indexing model from the issue text.