NVIDIA / NVIDIA/TensorRT-RTX

Does TensorRT_RTX support "MMCVModulatedDeformConv2d" ?

Open
#21 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
109
Forks
19
PR merge metrics
No merged PRs in 30d

Description

Hi all,

Currently I have an onnx model file
However, it contains one operator which is from mmcv as following:

Image

After executing ./tensorrt_rtx --onnx=srx2_fp32.onnx --saveEngine=srx2_RTX11_fp32.trt --computeCapabilities="89"

It shows the errors like:
Op type MMCVModulatedDeformConv2d is not supported in TensorRT-RTX

If I manually modify "MMCVModulatedDeformConv2d" into "DeformConv", which is supported by onnxruntime since opset=19, it will show the following errors:

[11/05/2025-03:26:33] [V] [TRT] Parsing node: DeformConv_1801 [DeformConv]
[11/05/2025-03:26:33] [V] [TRT] Searching for input: x.47
[11/05/2025-03:26:33] [V] [TRT] Searching for input: offset
[11/05/2025-03:26:33] [V] [TRT] Searching for input: mask
[11/05/2025-03:26:33] [V] [TRT] Searching for input: model.generator.deform_align.backward_1.weight
[11/05/2025-03:26:33] [V] [TRT] Searching for input: model.generator.deform_align.backward_1.bias
[11/05/2025-03:26:33] [V] [TRT] DeformConv_1801 [DeformConv] inputs: [x.47 -> (1, 96, 120, 160)[FLOAT]], [offset -> (1, 288, 120, 160)[FLOAT]], [mask -> (1, 144, 120, 160)[FLOAT]], [model.generator.deform_align.backward_1.weight -> (48, 96, 3, 3)[FLOAT]], [model.generator.deform_align.backward_1.bias -> (48)[FLOAT]],

Image

I build the custom operators as .so file and add it into onnxruntime sessionOptions and feed into InferenceSession.
And then I can infer the onnx model.

Does TensorRT_RTX support this or is there any methods to handle this?

Thanks

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the shown TensorRT-RTX command with the provided ONNX model and inspect the parser output for MMCVModulatedDeformConv2d and the manually renamed DeformConv node. Compare the operator inputs and the custom .so setup used with ONNX Runtime; done means establishing whether TensorRT-RTX supports this operator or documenting a viable handling method.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.