ROCm / ROCm/AMDMIGraphX

MIGraphX accuracy_checker problem when run the accuracy for different model(bert, gpt2)

Open
#2,181 3 comments 0 reactions 1 assignee View on GitHub

@CharlieL7 is already working on this.

Since Sep 22, 2023.

  • #2310 by @CharlieL7 — merged
Dominant language
C++
Stars
333
Forks
150
Avg merge
4d 19h
Merged PRs (30d)
54

Description

Description

When I run the accuracy_checker with this line without MLIR:
python accuracy_checker.py --fill1 --onnx /pathtomodel/bert_large_uncased_1_fp16_gpu.onnx

I get the message:
Outputs do not match
FAILED: MIGraphX is not within tolerance.

This is the output for this run line:
python accuracy_checker.py --fill1 --onnx /pathtomodel/bert_large_uncased_1_fp16_gpu.onnx --verbose

Output 0 is incorrect
Expected value:
[-0.90534556 -0.8867094 -0.05258729 ... 0.5805738 0.7309614
-0.9049778 ]
....
Actual value:
[-0.9003906 -0.8847656 0.0069809 ... 0.55908203 0.7246094
-0.9038086 ]

I add argument parser and migraphx.quantize_fp16 into python script and after that I think it's much better value but also failed.

This is the run line:
python accuracy_checker.py --fill1 --onnx /pathtomodel/bert_large_uncased_1_fp16_gpu.onnx --fp16 --verbose

This is the output:

Output 0 is incorrect ...
Expected value:
[-0.8792089 -0.8541642 0.2681412 ... 0.3513783 0.71706283
-0.9114238 ]
......
Actual value:
[-0.8730469 -0.85009766 0.28051758 ... 0.34179688 0.70458984
-0.90966797]

Tolerance in both case is 1e-3

I run also the bert_base_cased_1_fp16_gpu.onnx, distilgpt2_1_fp16_gpu.onnx and have the same messages.
Also try with enable MLIR but the messages is same.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.