Refactor parsing and handling of RNN to be compatible with internal quantizer
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 38/100
- Issue type
- Refactor
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- cpp
- Domain
- machine-learning
Research direction
Start by reproducing the failure with wlang_gru.onnx using the MIGraphX driver and FP8 options, then read the ONNX RNN specification and the existing rewrite_rnn pass. Review the op builder framework introduced in PR #3328. Done means wlang-gru and wlang-lstm models compile with FP8 quantization without the rnn_last_hs_output exception.
Written by the indexing model from the issue text.
Description
- Currently if you try to our FP8 quantizer on a model with RNN blocks you will hit an error like this:
Running [ MIGraphX Version: 2.13.0.2184b29e ]: /opt/rocm/bin/migraphx-driver perf wlang_gru.onnx --fill0 input.1 --onnx --fp8
Compiling ...
Reading: wlang_gru.onnx
terminate called after throwing an instance of 'migraphx::version_2_13_0::exception'
what(): /longer_pathname_so_that_rpms_can_support_packaging_the_debug_info_for_all_os_profiles/src/AMDMIGraphX/src/include/migraphx/operation.hpp:361: compute_op: Not computable: rnn_last_hs_output
- This was found for wlang-gru and wlang-lstm. Does not occur for int8 or int4 quantization because those only quantize GEMM and convolution instructions.
- This error will occur on any hardware.
- The issue comes from how RNN blocks are currently handled and how our quantizer uses the ref version of operators to calculate the quantization scales.
- The RNN operator comes from ONNX: https://onnx.ai/onnx/operators/onnx__RNN.html
- To fix this issue we need to refactor how the RNN operator is parsed and then rewritten in the
rewrite_rnnpass.- It was mentioned we could use the op builder framework introduced here to do the refactor: https://github.com/ROCm/AMDMIGraphX/pull/3328
- Dominant language
- C++
- Stars
- 333
- Forks
- 150
- Avg merge
- 4d 19h
- Merged PRs (30d)
- 54
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ROCm/AMDMIGraphX
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
ROCm/AMDMIGraphX#5101 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 62/100
ROCm/AMDMIGraphX#3592 ·
-
status: triage
ROCm/AMDMIGraphX#5237 · 1 assignee ·
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
ROCm/AMDMIGraphX#5235 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 42/100
ROCm/AMDMIGraphX#5234 · 1 comment · 1 reaction ·
All issues in ROCm/AMDMIGraphX
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
Sensor initialization takes very long when `--initial-sim-time` is set to current UNIX timestamp Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
gazebosim/gz-sensors#662 · 1 comment ·
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
comp-datalake
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
ClickHouse/ClickHouse#121222 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
LadybirdBrowser/ladybird#12123 ·