microsoft / microsoft/onnxruntime
[WebGPU EP] fp16 overflow in a decomposed LayerNorm: whisper-small q4f16 produces +Inf variance in the encoder and the decoder loops
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
`onnx-community/whisper-small` at dtype `q4f16` transcribes incorrectly on the WebGPU EP. A comma becomes a period, and from there the last sentence repeats until the token limit. The WASM EP and the CPU EP in the same package are correct on the same audio. An Intel Iris Xe (gen-12lp, D3D12) and an RTX 3050 Ti fail identically, so this is not a driver quirk.
What has kept it invisible is that nothing downstream looks wrong: `last_hidden_state` reports **zero Inf and zero NaN**, and the encoder `max|x|` only moves from 32.33 on CPU/WASM to 38.47 on WebGPU. You cannot find this by checking outputs for non-finite values.
The cause is that this export keeps LayerNorm **decomposed** — `ReduceMean / Sub / Pow / ReduceMean / Sqrt / Div`, every edge fp16 — and Whisper's residual stream carries large outliers. From layer 7 on, `|x - mean|` reaches ~795, and `Pow` squares that to ~630,000 against the f16 ceiling of 65,504. The overflow is then *absorbed*: an infinite variance makes the `Div` return 0, so the affected positions normalise to their bias alone and the tensor stays finite. The encoder output is quietly wrong rather than obviously broken.
Measured on the GPU by adding the intermediates as graph outputs and reading them back:
| tensor (encoder layer 7, `self_attn_layer_norm`) | +Inf count |
|---|---:|
| `Pow_output_0` | 92 |
| `ReduceMean_1_output_0` (the variance) | 49 |
so 49 of 1500 positions lose their content. Checking the same statistic offline on the real fp32 activations agrees: from layer 7 on, 94 elements spread over 47 of the 1500 rows exceed `sqrt(65504) = 255.9`; the worst single square is 651,870 (10x the ceiling) and the worst sum of squares 1,326,166 (20x).
Two controls, both through the EP's own `forceCpuNodeNames`, on the encoder:
| forced to CPU | nodes | encoder `max|x|` | decode |
|---|---:|---:|---|
| all `MatMulNBits` | 72 | 38.44 | still loops |
| `MatMul` | 24 | 38.44 | still loops |
| `Softmax` | 12 | 38.47 | still loops |
| `ReduceMean` + `Sub` + `Pow` + `Sqrt` + `Div` | 164 | **32.31** | **terminates at EOT** |
No single operator explains it — only the whole normalisation chain, which is what you would expect if the problem is the intermediate range rather than any one kernel.
CPU and WASM look correct only because ORT inserts fp32 casts around fp16 ops there, so the square is never actually computed in fp16.
One structural point that I think matters more than the model. **That LayerNorm is never fused.** The decomposed node names still exist at partition time — otherwise `forceCpuNodeNames` could not have matched them — so a six-op fp16 chain runs where a single kernel with f32 internals should. If the fusion covered this pattern, or if the WebGPU EP computed the mean/variance chain in f32 and cast at the end, the overflow would have nowhere to happen. That would also cover the other reports in this family, which are all "fp16 activations exceed 65504 somewhere in a normalisation".
Related, both closed by the stale bot rather than fixed: #26732 (Gemma-3 fp16/q4f16, where the overflow is in the residual stream itself and the Hub weights now ship `Clip` nodes as a model-side workaround) and #26367 (nanochat q4f16, same symptom, never attributed). This is a third model, and here the root cause is measured rather than inferred.
To be explicit, since it came up on #29611 where I first reported this: **it is not a MatMul accumulator overflow, and #29599 would not fix it.**
### To reproduce
1. `onnx-community/whisper-small`, `onnx/encoder_model_q4f16.onnx` + the merged decoder, on `onnxruntime-web` 1.29.0-dev (the wasm+webgpu package, not jsep).
2. Any speech of ~10 s or more; I used the standard 11 s JFK sample and a 30 s clip. Greedy decoding, `language: "en"`.
3. Compare against the same package on the WASM EP, or against the CPU EP. WebGPU changes punctuation and then repeats the final sentence to the token limit.
4. To see the cause rather than the symptom, add `/layers.7/self_attn_layer_norm/Pow_output_0` and the following `ReduceMean` output to the encoder's graph outputs and count non-finite values.
Workaround for anyone hitting this today: pass the normalisation chain in `forceCpuNodeNames` as a **WebGPU EP option** (`executionProviders: [{ name: 'webgpu', forceCpuNodeNames: [...] }]`), which restores correct output at an obvious performance cost. Note it has to be an EP option — passing it as a session config entry silently logs `force CPU node count: 0` and has a 8192-character cap per value.
### Urgency
Not blocking; there is a working fallback in `q4`. But `q4f16` is the dtype users are pointed at for WebGPU, the failure is silent, and it reads as a model quality problem rather than a runtime bug, so I expect it is being hit and not reported.
### ONNX Runtime Installation
Released Package
### ONNX Runtime Version or Commit ID
1.29.0-dev (onnxruntime-web, wasm+webgpu build)
### Execution Provider
'webgpu' (WebGPU)
Contributor guide
Research direction
Reproduce with onnx-community/whisper-small using encoder_model_q4f16.onnx and the merged decoder on the WebGPU EP, comparing against WASM or CPU. Add /layers.7/self_attn_layer_norm/Pow_output_0 and the following ReduceMean output as graph outputs, then inspect the WebGPU EP's handling of the decomposed normalization chain. Done means the WebGPU result matches the reference without the silent overflow and decoding loop.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, wasm
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100