intel / intel/torch-xpu-ops

Torchbench model functorch_dp_cifar10 got fail_accuracy

Open
#1,877 2 comments 0 reactions 1 assignee Claimed by @DamJanusz View on GitHub
Accuracy hw: BMG hw: PVC
Dominant language
Python
Stars
113
Forks
128
Avg merge
5d 9h
Merged PRs (30d)
112

Description

### 🐛 Describe the bug

Torchbench model `squeezenet1_1`(bfloat16_inference) and `functorch_dp_cifar10` (float16_train/float32_train/bfloat16_train) got fail_accuracy.

```
C:\pytorch>python benchmarks\dynamo\torchbench.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --inference --bfloat16 --only squeezenet1_1
loading model: 0it [00:01, ?it/s]
xpu eval squeezenet1_1
E0723 13:39:24.765000 8228 site-packages\torch\_dynamo\utils.py:2998] RMSE (res-fp64): 0.06339, (ref-fp64): 0.01168 and shape=torch.Size([4, 1000]). res.dtype: torch.bfloat16, multiplier: 3.000000, tol: 0.000100, use_larger_multiplier_for_smaller_tensor: 0
fail_accuracy

C:\pytorch>python benchmarks\dynamo\torchbench.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --training --bfloat16 --only functorch_dp_cifar10
loading model: 0it [00:01, ?it/s]
xpu train functorch_dp_cifar10
E0723 13:41:50.568000 3036 site-packages\torch\_dynamo\utils.py:2998] RMSE (res-fp64): 0.00109, (ref-fp64): 0.00027 and shape=torch.Size([64]). res.dtype: torch.bfloat16, multiplier: 3.000000, tol: 0.001000, use_larger_multiplier_for_smaller_tensor: 0
E0723 13:42:38.554000 3036 site-packages\torch\_dynamo\utils.py:2859] Accuracy failed for key name bn1.bias.grad
fail_accuracy

```

### Versions

- Both PT2.8 (03eb1e40f9ddf09cb9eef86ace74332e87f11a79, v2.8.0-rc2) and PT2.7(b1940b5867e40e40ebdce4db76f76d3d0b71d3f4, v2.7.0-rc2) had this issue (BMG Windows).

- Driver: 32.0.101.6913

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.