ROCm / ROCm/AMDMIGraphX

Accuracy issues with ONNX Zoo QDQ models

Open
#2,704 2 comments 0 reactions 2 assignees View on GitHub

@TedThemistokleous is already working on this.

Since Mar 6, 2024.

bug onnx
Dominant language
C++
Stars
333
Forks
150
Avg merge
4d 19h
Merged PRs (30d)
54

Description

12 quantized models from ONNX Zoo fails with dirver verify command. Some models are showing extreme differences between ref and gpu results.

rocm version: 5.7.0.50700-63~20.04
migraphx version: 352dcea2c6a03c495a6ba8667e19811bc5d1399b

RMS Error: -nan
Max diff: inf
Mismatch at 0: 0.00336882 != -inf

RMS Error: 0.299546
Max diff: 52
Mismatch at 0: 1 != 18

RMS Error: 0.982469
Max diff: 0.984108
Mismatch at 0: 0.0948163 != 1
RMS Error: 0.310712
Max diff: 1.75075e+06
Mismatch at 0: -0.43521 != -160776
RMS Error: 0.230274
Max diff: 9.39427e+08
Mismatch at 0: -0.00747727 != -2.52037e+08
RMS Error: 0.337116
Max diff: 1.45539e+09
Mismatch at 0: -2.89145 != 5.29669e+08
RMS Error: 0.206697
Max diff: 7.68712e+06
Mismatch at 0: -0.607095 != -769524
RMS Error: 0.410436
Max diff: 1517.48
Mismatch at 0: 7.7691 != -1509.71

RMS Error: 0.360478
Max diff: 287.336
Mismatch at 0: 6.21395 != -281.122

The remaining 6 with not that big difference (<1): bvlcalexnet-12-qdq.onnx, efficientnet-lite4-11-qdq, googlenet-12-qdq.onnx, inception-v1-12-qdq.onnx, squeezenet1.0-13-qdq.onnx, zfnet512-12-qdq.onnx

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.