apache / apache/tvm

[Bug][Relax] Import failures on PaddleOCR and FasterRCNN-style models: Squeeze axes, shape Gather, and dynamic TopK

Open
#19,965 2 comments 0 reactions 0 assignees View on GitHub
needs-triage type: bug
Dominant language
Python
Stars
13.7k
Forks
4k
Avg merge
2d 1h
Merged PRs (30d)
112

Description

## Summary

During compatibility testing, two industrial-style ONNX graphs consistently fail at the TVM Relax ONNX importer stage, before tuning can start:

1. `PP-OCRv6_tiny.onnx`, exported from a PaddleOCR-style model, fails around `Squeeze -> Transpose`.
2. `FasterRCNN-12.onnx`, a detection graph with dynamic post-processing, fails around shape `Gather` and then dynamic `TopK`.

I would like to ask whether these are expected importer limitations in the current Relax ONNX frontend, and whether the preprocessing workarounds described below are recommended, or if there is a better official way to handle these graphs.

## Environment

```text
TVM version: 0.24.0
Python: 3.11.15
ONNX: 1.21.0
OS: Linux
Frontend used: tvm.relax.frontend.onnx.from_onnx
```

## Case 1: PP-OCRv6_tiny Squeeze axes are lost before Transpose

The model is `PP-OCRv6_tiny.onnx`, opset 11. The failing pattern in the ONNX graph is:

```text
Squeeze.0
inputs = ['p2o.pd_op.pool2d.0.0']
outputs = ['p2o.pd_op.squeeze.0.0']
attrs = {'axes': [2]}

Transpose.0
inputs = ['p2o.pd_op.squeeze.0.0']
outputs = ['p2o.pd_op.transpose.0.0']
attrs = {'perm': [0, 2, 1]}
```

Using either OCR runtime shape below gives the same failure:

```python
from tvm.relax.frontend.onnx import from_onnx
import onnx

model = onnx.load("PP-OCRv6_tiny.onnx")
from_onnx(
model,
shape_dict={"x": [1, 3, 48, 128]},
dtype_dict={"x": "float32"},
opset=11,
keep_params_in_input=False,
)
```

Observed error:

```text
Error converting operator Transpose, with inputs: [R.squeeze(lv151, axis=None)]
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)
```

The ONNX node has `Squeeze axes=[2]`, so the expected output rank should be 3 and `Transpose perm=[0,2,1]` should be valid. The TVM error shows `R.squeeze(..., axis=None)`, which suggests that the importer may be ignoring the ONNX `axes` attribute for this opset/node form.

### Related PaddleOCR Conv pattern

The same model also contains Paddle-style grouped/depthwise `1xK` conv patterns:

```text
node = Conv.35
inputs = ['p2o.pd_op.unsqueeze.1.0', 'p2o.pd_op.unsqueeze.0.0']
outputs = ['p2o.pd_op.depthwise_conv2d.9.0']
attrs = {
'dilations': [1, 1],
'kernel_shape': [1, 5],
'strides': [1, 1],
'group': 160,
'pads': [0, 2, 0, 2]
}
```

I experimented with two graph rewrites:

1. Normalize symmetric ONNX Conv pads from `[top,left,bottom,right]` to `[h,w]` for TVM.
2. Rewrite `Unsqueeze(axis=2) -> Conv(1xK) -> Squeeze(axis=2)` into an equivalent Conv1D form.

However, the first rewrite is not ONNX-spec-safe as a persisted ONNX model, because ONNX Runtime validation reports:

```text
Node (Conv.0) Op (Conv) [ShapeInferenceError] Attribute pads has incorrect size
```

So I currently treat this as a TVM-only workaround and avoid saving it as a general runtime ONNX graph.

## Case 2: FasterRCNN-12 dynamic detection post-processing

The model is `FasterRCNN-12.onnx`, opset 12. It contains a full detection post-processing graph:

```text
TopK: 7
NonMaxSuppression: 85
RoiAlign: 4
Resize: 3
Shape: 117
Gather: 798
```

Raw import fails first at shape `Gather`:

```text
Error converting operator Gather, with inputs: [R.shape([1, 3, 1, 1]), 2034]
AssertionError: Only constant indices supported for shape gather.
```

After freezing/stabilizing static shape subgraphs and importing the prepared runtime ONNX with:

```python
from_onnx(
model,
shape_dict={"image": [3, 224, 224]},
dtype_dict={"image": "float32"},
opset=12,
keep_params_in_input=False,
)
```

the next importer failure is dynamic `TopK`:

```text
Error converting operator TopK, with inputs: [R.sigmoid(lv242), v_5635]
ValueError: TopK k must be a constant
```

## Current workarounds in my pipeline

The pipeline currently does the following:

- Fold static shape subgraphs into initializers when this preserves ONNX Runtime validation.
- Fix missing or empty `Resize` ROI inputs.
- Rewrite static `Split` tensor inputs to `Slice` where TVM treats the second input as dynamic.
- Remove or bypass inference-time `Dropout`.
- Skip full detection post-processing graphs containing `NonMaxSuppression + dynamic TopK`.
- Skip Paddle-style degenerate grouped `1xK` conv graphs when the rewrite would make the persisted ONNX invalid.

## Questions

1. For opset 11 `Squeeze` with an `axes` attribute, should the Relax ONNX importer preserve the axes and emit `R.squeeze(..., axis=[2])` instead of `axis=None`?
2. Is the shape `Gather` failure expected when the input is `R.shape(...)` and the index is a scalar constant-like value?
3. Is dynamic `TopK k` unsupported by design in Relax ONNX import, or is there a recommended way to keep it symbolic?
4. For full detection graphs with `NonMaxSuppression + TopK`, does the TVM team recommend splitting the graph before import, or should users expect full-graph import to work eventually?
5. Are graph-level rewrites such as static shape folding, static Split-to-Slice, and `Unsqueeze-Conv-Squeeze -> Conv1D` considered reasonable preprocessing for TVM, or is there a more official path?

## Expected behavior

Ideally, TVM should import the valid ONNX graph or report a precise unsupported-pattern diagnostic. In the PP-OCRv6 case, the `Squeeze axes=[2]` attribute appears to be valid and should not reduce the tensor with `axis=None`.

## Attachments

log:

```text
[11:04:07] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66: Warning: BlockBuilder destroyed with remaining blocks!
[11:04:07] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66: Warning: BlockBuilder destroyed with remaining blocks!
[11:04:07] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66: Warning: BlockBuilder destroyed with remaining blocks!
[11:04:08] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66: Warning: BlockBuilder destroyed with remaining blocks!

==========================================================================================
PP-OCRv6_tiny raw ONNX -> TVM Relax from_onnx
==========================================================================================
shape_dict = {'x': [1, 3, 48, 1]}
dtype_dict = {'x': 'float32'}
Error converting operator Transpose, with inputs: [R.squeeze(lv151, axis=None)]
IMPORT_FAILED
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5173, in from_onnx
self._construct_nodes(graph)
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5382, in _construct_nodes
raise err
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5376, in _construct_nodes
op = self._convert_operator(op_name, inputs, attr, self.opset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5476, in _convert_operator
sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 931, in _impl_v13
raise ValueError(
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)
[Preprocess] Normalized 37 symmetric Conv pads from 4D to 2D for TVM.

==========================================================================================
PP-OCRv6_tiny after symmetric Conv pads normalization
==========================================================================================
shape_dict = {'x': [1, 3, 48, 1]}
dtype_dict = {'x': 'float32'}
Error converting operator Transpose, with inputs: [R.squeeze(lv151, axis=None)]
IMPORT_FAILED
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5173, in from_onnx
self._construct_nodes(graph)
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5382, in _construct_nodes
raise err
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5376, in _construct_nodes
op = self._convert_operator(op_name, inputs, attr, self.opset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5476, in _convert_operator
sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 931, in _impl_v13
raise ValueError(
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)
[Preprocess] Normalized 37 symmetric Conv pads from 4D to 2D for TVM.

==========================================================================================
PP-OCRv6_tiny after pads normalization + Unsqueeze-Conv-Squeeze to Conv1D rewrite
==========================================================================================
shape_dict = {'x': [1, 3, 48, 1]}
dtype_dict = {'x': 'float32'}
Error converting operator Transpose, with inputs: [R.squeeze(lv151, axis=None)]
IMPORT_FAILED
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5173, in from_onnx
self._construct_nodes(graph)
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5382, in _construct_nodes
raise err
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5376, in _construct_nodes
op = self._convert_operator(op_name, inputs, attr, self.opset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5476, in _convert_operator
sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 931, in _impl_v13
raise ValueError(
ValueError: Transpose: number of axes in perm attribute (3) must equal the number of input tensor dimensions (2)

==========================================================================================
PP-OCRv6_tiny representative grouped 1xK Conv nodes
==========================================================================================
node= Conv.35 inputs= ['p2o.pd_op.unsqueeze.1.0', 'p2o.pd_op.unsqueeze.0.0'] outputs= ['p2o.pd_op.depthwise_conv2d.9.0'] attrs= {'dilations': [1, 1], 'kernel_shape': [1, 5], 'strides': [1, 1], 'group': 160, 'pads': [0, 2, 0, 2]}

==========================================================================================
FasterRCNN-12 raw ONNX -> TVM Relax from_onnx
==========================================================================================
shape_dict = {'image': [3, 1, 1]}
dtype_dict = {'image': 'float32'}
Error converting operator Gather, with inputs: [R.shape([1, 3, 1, 1]), 2034]
IMPORT_FAILED
AssertionError: Only constant indices supported for shape gather.
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5173, in from_onnx
self._construct_nodes(graph)
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5382, in _construct_nodes
raise err
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5376, in _construct_nodes
op = self._convert_operator(op_name, inputs, attr, self.opset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5476, in _convert_operator
sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 1086, in _impl_v13
assert isinstance(indices, relax.Constant), (
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Only constant indices supported for shape gather.

==========================================================================================
FasterRCNN-12 prepared runtime_fp32.onnx -> TVM Relax from_onnx
==========================================================================================
shape_dict = {'image': [3, 224, 224]}
dtype_dict = {'image': 'float32'}
Error converting operator TopK, with inputs: [R.sigmoid(lv242), v_5635]
IMPORT_FAILED
ValueError: TopK k must be a constant
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5173, in from_onnx
self._construct_nodes(graph)
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5382, in _construct_nodes
raise err
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5376, in _construct_nodes
op = self._convert_operator(op_name, inputs, attr, self.opset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 5476, in _convert_operator
sym = op_function(self.bb, inputs, attrs, [self._nodes, self._params])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/perception/Nanmur/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 4154, in _impl_v11[11:04:08] /home/perception/Nanmur/tvm/src/relax/ir/block_builder.cc:66: Warning: BlockBuilder destroyed with remaining blocks!

raise ValueError("TopK k must be a constant")
ValueError: TopK k must be a constant

==========================================================================================

```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with python/tvm/relax/frontend/onnx/onnx_frontend.py, especially the Squeeze, Gather, and TopK conversion paths shown in the traceback. Run the provided from_onnx reproducers for PP-OCRv6_tiny.onnx and FasterRCNN-12.onnx, then determine whether these patterns can be imported or need explicit unsupported-pattern diagnostics. Done means the valid Squeeze case imports correctly and the remaining Gather or dynamic TopK limitations are reported precisely.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.