Worsened inference performance on ONNX for models with higher resolution than 800x1333

Ask a Question

Question

Hi there,

I have converted a Mask-RCNN model from MMDetection to ONNX. I have tested it both to run it on ONNX-Runtime and also converted it further to TensorRT for speedup. (I am using a Jetson AGX Xavier)
To conduct the inference on ort, I have followed this this script closely. It works fine for smaller image sizes such as 480x640 but I have tried using 3MP(1544x2064) images and the performance drops drastically.

The page also states the following: "This model can take images of different sizes as input. However, to achieve best performance, it is recommended to resize the image such that both height and width are within the range of [800, 1333], and then pad the image with zeros such that both height and width are divisible by 32."

So why does the performance drop for images that are larger than [800, 1333]? I would really like to use images with higher resolutions. When I run inference on MMDetection (pytorch) with 3MP images, the accuracy of the predictions is much better than with ONNX (but the inference is quite slow so I don't want to use it).

When I convert the ONNX model further to TensorRT and run inference through TensorRT, the fps increases further (as expected) but the problem with higher resolutions resulting in faultier predictions remains.

So I am guessing there is some kind of issue with the conversion to ONNX...
I'd be very thankful if anyone could give me some insight to this issue!

Is this issue related to a specific model?
Model name (e.g. mnist): Mask-RCNN with ResNet-50 Backbone
Model opset (e.g. 7): Opset 11

Start with the ONNX models vision/object_detection_segmentation/mask-rcnn script and compare its resizing and padding behavior for 480x640 versus 1544x2064 inputs. Reproduce the result with the Mask-RCNN ResNet-50 model at opset 11, then compare ONNX Runtime, TensorRT, and MMDetection outputs to determine whether conversion or input sizing causes the performance and prediction differences.