Exported Mask-RCNN to ONNX produces wrong results

🐛 Bug

I exported the pretrained rcnn model to onnx via

model_tv = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True)
model_tv.eval()
torch.onnx.export(model_tv, torch.rand(1,3,800,800), "mask_rcnn_r50_fpn.onnx",
                  do_constant_folding=True,
                  opset_version=12  # opset_version 11 required for Mask R-CNN
                  )

However, when I perform inference on that model:

import onnxruntime as rt
import numpy
import argparse
import numpy as np
import cv2

parser = argparse.ArgumentParser(description='ImageNet native ORT')
parser.add_argument('--model', type=str, required=True)
parser.add_argument('--image', type=str, required=True)
args, args_other = parser.parse_known_args()

sess = rt.InferenceSession(args.model)
input_name = sess.get_inputs()[0].name

def load_image(img_path):
    # CV loads in BGR, and rcnn expects rgb
    loaded = cv2.imread(img_path)
    loaded = cv2.cvtColor(loaded, cv2.COLOR_BGR2RGB)
    img_data = loaded.transpose(2, 0, 1)

    # The mean values provided are in RGB format
    mean_vec = np.array([0.485, 0.456, 0.406])
    stddev_vec = np.array([0.229, 0.224, 0.225])

    norm_img_data = np.zeros(img_data.shape).astype('float32')
    for i in range(img_data.shape[0]):  
        norm_img_data[i,:,:] = (img_data[i,:,:]/255 - mean_vec[i]) / stddev_vec[i]
    norm_img_data = np.expand_dims(norm_img_data, axis=0)
    return norm_img_data

def perform_inference(file):
    preprocessed = load_image(file)
    print(file)
    result = sess.run(None, {input_name: preprocessed})
    import pdb; pdb.set_trace()


perform_inference(args.image)

with the attached image

The results are incorrect:

I assumed the labels are the standard Coco labels as provided e.g. here

I'm fairly certain this is either a preprocessing issue or an issue of using the wrong label set, but as far as I could find both are correct.

In addition to the labels being mismatched, the probabilities are a lot lower than I got using the mask-rcnn model from the onnx model zoo

Start by reproducing the export from torchvision.models.detection.maskrcnn_resnet50_fpn and compare its ONNX Runtime output with the native model. Check the preprocessing, COCO label assumptions, and the exported model's outputs; done means the labels and confidence scores agree with the expected Mask R-CNN results.