microsoft / microsoft/onnxruntime

[WebGPU EP] Deterministic audio corruption in Kokoro-82M TTS on Intel Iris Xe — fp32 corrupts a subset of inputs, fp16 corrupts all; same model clean on WASM EP and on Qualcomm Adreno

Open
#29,807 4 comments 0 reactions 0 assignees View on GitHub
ep:WebGPU model:transformer platform:mobile platform:web stale
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

Running the Kokoro-82M TTS model (`onnx-community/Kokoro-82M-v1.0-ONNX`) on the WebGPU EP
on **Intel Iris Xe** produces **deterministically corrupted audio** for certain inputs:

- **fp32**: roughly ⅓ of sentences render with a degraded "congested" timbre — measured as
either elevated high-frequency noise (6kHz+ RMS share 0.19–0.42 vs ≤0.15 for clean renders
of the same sentences) or mid-band (700–2000Hz) energy dropout.
- **fp16**: **every** input we tested renders corrupted (10/10 sentences, HF share 0.26–0.38
vs 0.13–0.31 for paired WASM renders of the identical text).
- **q8 / q4f16** on WebGPU produce unintelligible output (phoneme garbage / noise) and q8 is
also slower than the WASM EP — possibly a separate issue, included for completeness.

Key characteristics that should help triage:

1. **Deterministic and content-triggered**: the same input sentence produces byte-identical
corrupt audio on every run (verified across independent sessions — identical per-sentence
RMS statistics). Other sentences through the same session render perfectly. This is not a
race/uninitialized-memory pattern; it looks like a numerical defect in specific kernel(s)
exercised by specific input sequences.
2. **Hardware-specific**: the identical code + model + dtype on **Qualcomm Adreno 7xx**
(Android, same Chromium 150) renders **completely clean** at fp32. The **WASM EP** on the
affected machine is also completely clean.
3. **Timbre-level, not logic-level**: the corrupted audio remains 100% intelligible to
whisper-large-v3 (perfect transcription) but is audibly degraded to humans — consistent
with bounded numerical error rather than wrong control flow.

## To reproduce

```js
import { KokoroTTS } from "kokoro-js"; // 1.2.0 (bundles @huggingface/transformers 3.8.1 / ort-web 1.22.x)
const tts = await KokoroTTS.from_pretrained("onnx-community/Kokoro-82M-v1.0-ONNX", {
device: "webgpu", dtype: "fp32", // also try "fp16"
});
// Sentences that reproduce the fp32 corruption deterministically on Intel Iris Xe:
const bad = "Each sentence you hear was generated separately, so the timing log shows exactly how long every line took.";
const good = "The new engine runs on the graphics card instead.";
const audio = await tts.generate(bad, { voice: "af_bella" });
// Compare against device:"wasm" render of the same text: HF-noise share ~2x higher, audibly "congested".
```

**Attached: a clean/corrupt pair of the IDENTICAL sentence** ("Each sentence you hear was
generated separately, so the timing log shows exactly how long every line took."):
- `ort-issue-CORRUPT-intel-webgpu-fp32.mp4` — WebGPU EP fp32 render on Intel Iris Xe
- `ort-issue-CLEAN-wasm-same-sentence.mp4` — WASM EP render of the same text, same machine
The corrupt render reproduces byte-identically on every run. A minimal standalone repro page
(single HTML, kokoro-js from CDN + device/dtype picker + spectral scoring) available on request.

## Environment

- onnxruntime-web **1.22.x** (as bundled by @huggingface/transformers 3.8.1 / kokoro-js 1.2.0)
- **Affected:** Windows 11, Chromium 150 (Chrome and Brave), Intel **Iris Xe** (i7-1360P, Gen-12LP)
- **Not affected:** same code/model on Android Chromium 150, Qualcomm **Adreno 7xx** (clean at fp32);
WASM EP clean everywhere (q8 and fp32)
- crossOriginIsolated=true (threaded WASM available); issue reproduces regardless of isolation

## Expected behavior

WebGPU EP output numerically consistent with WASM EP output for the same model/dtype
(within normal float tolerance — not audible timbre corruption).

## Additional context

We measured the corruption's spectral transfer (clean-vs-corrupt renders of identical text,
deterministic pairing): mid-band 300–2000Hz attenuated ~1.2–1.6dB, 4.5kHz+ boosted
+1.3–1.8dB with added noise (hiss variant), plus a second variant with mid-band dropout
(muffle). Happy to provide audio pairs, per-sentence spectral data, or run diagnostics
builds on the affected hardware.

https://github.com/user-attachments/assets/f3ccd31d-8c33-4073-b634-bc71da306ee6
https://github.com/user-attachments/assets/50faa1f0-3f21-44f4-ac96-4645d9d31c8f

Contributor guide

Open the contributing guide

Research direction

Start by running the provided kokoro-js reproduction on Intel Iris Xe with the WebGPU and WASM EPs, comparing fp32 and fp16 output for the listed good and bad sentences. Use the attached clean/corrupt pair and spectral measurements while narrowing the issue to the affected WebGPU execution path; done means WebGPU output is numerically consistent with the WASM reference without audible corruption.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
ai-infra-agents, machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.