ROCm / ROCm/AMDMIGraphX

[onnx_parser][resolved: invalid model] slanet-plus.onnx contains prohibited outer-scope name shadowing

Open
#5,078 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
333
Forks
150
Avg merge
4d 19h
Merged PRs (30d)
54

Description

[!IMPORTANT]
Correction / resolution: The original report incorrectly stated that the overlapping names were valid ONNX and that nested-subgraph inputs were independently scoped. Following @pfultz2's correction and re-verification against the current ONNX IR specification, that interpretation is withdrawn.

ONNX nested subgraphs are lexically scoped, and variable shadowing of names visible from an outer scope is prohibited. slanet-plus.onnx violates this requirement. MIGraphX's rejection at check_sorted / parse_inputs is therefore correct, and this issue is no longer requesting that MIGraphX relax the check.

Summary

MIGraphX fails to compile a valid ONNX model that contains a Loop (control-flow) op whose body subgraph declares input parameters whose names also appear in the enclosing graph. This name overlap is permitted by the ONNX specification — subgraph inputs are independently scoped — but check_sorted in src/AMDMIGraphX/src/onnx/onnx_parser.cpp:498 treats it as a conflict and aborts parsing:

check_sorted: subgraph "PaddlePaddle Graph 1" has parameter name "gru_cell_0.w_1" existing in parent graph!

The pattern is produced routinely by Paddle2ONNX when exporting GRU / recurrent layers: weight names such as gru_cell_0.w_1 are emitted once in the parent graph and passed into the Loop body, so the same name legitimately appears in both scopes. As a result MIGraphX cannot run a whole class of Paddle-derived models (table / OCR / recurrent).

Environment

Component Version
MIGraphX 2.15.0.20250912-17-200-gde19b73ad (migraphx-driver --version)
ROCm 7.2.1
GPU AMD gfx1100 (Radeon PRO W7900, RDNA3)
ONNX Runtime 1.23.2, via the MIGraphX EP (AMD wheel onnxruntime-migraphx, repo.radeon.com/rocm/manylinux/rocm-rel-7.2.1/)
Model slanet-plus.onnx (SLANet-Plus wireless table-structure recognition); ONNX opset 14, IR version 7, exported by Paddle2ONNX

The model is public: opendatalab/PDF-Extract-Kit-1.0models/TabRec/SlanetPlus/slanet-plus.onnx (~7.4 MB).

Reproducer

Download the model:

huggingface-cli download opendatalab/PDF-Extract-Kit-1.0 \
  models/TabRec/SlanetPlus/slanet-plus.onnx --local-dir .

Run (the input shape is not load-bearing — parsing fails before input validation):

import onnxruntime as ort, numpy as np
sess = ort.InferenceSession(
    "models/TabRec/SlanetPlus/slanet-plus.onnx",
    providers=["MIGraphXExecutionProvider", "CPUExecutionProvider"],
)
x = np.random.randn(1, 3, 488, 488).astype(np.float32)
out = sess.run(None, {sess.get_inputs()[0].name: x})   # raises RuntimeException

Expected behavior

MIGraphX compiles and runs the model. The model is valid ONNX:

  • onnx.checker.check_model(model) passes.
  • The same model runs correctly under the CPU EP — providers=["CPUExecutionProvider"] returns outputs with shapes (1, 10, 8) and (1, 10, 50).

Actual behavior

RuntimeException raised from session.run():

migraphx_parse_onnx_buffer: Error: /.../src/AMDMIGraphX/src/onnx/onnx_parser.cpp:498:
  check_sorted: subgraph "PaddlePaddle Graph 1" has parameter name "gru_cell_0.w_1"
  existing in parent graph!
[E:onnxruntime:, sequential_executor.cc:572 ExecuteKernel] Non-zero status code returned
  while running MGXKernel_graph_PaddlePaddle Graph 0_8504660414787792506_0 node.
  Name:'MIGraphXExecutionProvider_MGXKernel_graph_PaddlePaddle Graph 0_8504660414787792506_0_0'
  Status Message: Failed to call function
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 :
  RUNTIME_EXCEPTION ... Status Message: Failed to call function

Root-cause analysis

  • The model contains exactly one ONNX Loop op (opset 14); its body subgraph is named "PaddlePaddle Graph 1" (Paddle2ONNX export of a GRU). The body declares 32 inputs.
  • 16 of those 32 body inputs share their names with node outputs in the parent graph — e.g. gru_cell_0.w_1, gru_cell_0.w_0, gru_cell_0.b_0, gru_cell_0.b_1, linear_*.w_0, linear_*.b_0. These are the GRU/linear weight names, emitted once in the parent graph and passed into the Loop body.
  • This overlap is legal ONNX. A subgraph's (Loop body's) inputs are independently scoped; their names are local to the body and need not be unique with respect to the enclosing graph — outer values are bound to body inputs by position at the Loop node, not by name uniqueness. Concrete evidence the model is not malformed:
    • onnx.checker.check_modelPASS.
    • The CPU EP runs it and produces correct output shapes.
  • check_sorted (onnx_parser.cpp:498) walks the subgraph's parameter names and rejects any that also occur in the parent graph — an invariant not required by the ONNX spec — so it rejects a valid model.
  • The control-flow subgraph is the trigger. Two sibling table models from the same package — unet.onnx (UNet table line recovery) and PP-LCNet_x1_0_table_cls.onnx (table wired/wireless classifier) — contain no Loop and compile/run on MIGraphX with 100% node coverage and bit-exact outputs vs CPU (UNet ~20× faster than CPU, PP-LCNet ~12× faster). Only the Loop-containing slanet-plus.onnx fails.

Suggested fix

Honor ONNX subgraph scoping in the parser: a Loop/If/Scan body-input name that coincides with an enclosing-graph name is not a conflict and must not abort parsing (check_sorted / the subgraph parameter handling in onnx_parser.cpp). If some internal MIGraphX representation cannot tolerate the overlap, rename-on-import rather than rejecting the model.

Workaround

Force the affected model onto the CPU EP: providers=["CPUExecutionProvider"] (correct; forfeits the GPU speedup). Models without control-flow ops are unaffected.

References

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/AMDMIGraphX/src/onnx/onnx_parser.cpp:498 and the check_sorted/parse_inputs path. Re-run the listed slanet-plus.onnx reproducer, then compare the issue’s correction with the current ONNX IR specification and onnx.checker result. Because the correction withdraws the original report, done means establishing whether any actionable parser change remains.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.