microsoft / microsoft/onnxruntime

[ORT GPU (DML)][WebNN] Fail to run two WebNN batchNormalization tests due to tolerance issues

Open
#26,532 1 comment 0 reactions 1 assignee Claimed by @fdwr View on GitHub
ep:DML ep:WebNN WebNN
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

Fail to run below two WebNN batchNormalization tests due to tolerance issues by ORT default GPU DML EP.

|Test|Error message|
|---|--|
batchNormalization float32 2D constant tensors default options | assert_less_than_equal: assert_array_approx_equals_ulp: test batchNormalization float32 actual 1.8526287078857422 should be close enough to expected 1.8526301383972168 by ULP distance: expected a number less than or equal to 6n but got 12n
batchNormalization options.axis=0 + gelu|assert_less_than_equal: assert_array_approx_equals_ulp: test batchNormalization gelu float32 actual -0.035951320081949234 should be close enough to expected -0.03595117852091789 by ULP distance: expected a number less than or equal to 24n but got 38n

/cc @fdwr PTAL, thanks!

### To reproduce

1. Install Windows App SDK Stable [1.8.2 (1.8.251003001)](https://learn.microsoft.com/en-us/windows/apps/windows-app-sdk/stable-channel#version-18)
2. Install [lates Edge Canary browser](https://www.microsoft.com/en-us/edge/download/insider?form=MA13FJ)
3. Launch Edge Canary browser, navigate to `about://flags` enabled "Enables WebNN API" and "ONNX Runtime backend for WebNN" flags, then relaunch browser
4. Run each failure test using below test url

|Test| Test URL|
|-- | --|
batchNormalization float32 2D constant tensors default options | https://wpt.live/webnn/conformance_tests/batch_normalization_constant.https.any.html?device=gpu&tc=batchNormalization%20float32%202D%20constant%20tensors%20default%20options
batchNormalization options.axis=0 + gelu|https://wpt.live/webnn/conformance_tests/subgraph.https.any.html?device=gpu&tc=batchNormalization%20options.axis=0%20%2B%20gelu

### Urgency

_No response_

### ONNX Runtime Installation

Released Package Windows App SDK Stable [1.8.2 (1.8.251003001)](https://learn.microsoft.com/en-us/windows/apps/windows-app-sdk/stable-channel#version-18)

### ONNX Runtime Version or Commit ID

1.23.25.928

### Execution Provider

ORT default GPU EP

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.