MIGraphX accuracy_checker problem when run the accuracy for different model(bert, gpt2)
- Dominant language
- C++
- Stars
- 333
- Forks
- 150
- Avg merge
- 4d 19h
- Merged PRs (30d)
- 54
Description
Description
When I run the accuracy_checker with this line without MLIR:
python accuracy_checker.py --fill1 --onnx /pathtomodel/bert_large_uncased_1_fp16_gpu.onnx
I get the message:
Outputs do not match
FAILED: MIGraphX is not within tolerance.
This is the output for this run line:
python accuracy_checker.py --fill1 --onnx /pathtomodel/bert_large_uncased_1_fp16_gpu.onnx --verbose
Output 0 is incorrect
Expected value:
[-0.90534556 -0.8867094 -0.05258729 ... 0.5805738 0.7309614
-0.9049778 ]
....
Actual value:
[-0.9003906 -0.8847656 0.0069809 ... 0.55908203 0.7246094
-0.9038086 ]
I add argument parser and migraphx.quantize_fp16 into python script and after that I think it's much better value but also failed.
This is the run line:
python accuracy_checker.py --fill1 --onnx /pathtomodel/bert_large_uncased_1_fp16_gpu.onnx --fp16 --verbose
This is the output:
Output 0 is incorrect ...
Expected value:
[-0.8792089 -0.8541642 0.2681412 ... 0.3513783 0.71706283
-0.9114238 ]
......
Actual value:
[-0.8730469 -0.85009766 0.28051758 ... 0.34179688 0.70458984
-0.90966797]
Tolerance in both case is 1e-3
I run also the bert_base_cased_1_fp16_gpu.onnx, distilgpt2_1_fp16_gpu.onnx and have the same messages.
Also try with enable MLIR but the messages is same.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.