facebookresearch / facebookresearch/detectron2

Issue with Onnx'd detectron2 mask-rcnn bounding boxes

Open
#5,490 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

I have build a mask-rcnn in detectron2 with `COCO-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_3x.yaml` as a back bone but have a bounding box miss match compared to the original image

When using the detectron2 model through the inbuilt methods like so:
```python
cfg = get_cfg()
# Use a pre-trained Mask R-CNN model from the model zoo
cfg.merge_from_file("../../outputs/models/config_custom_2025-09-13 05:57:14.510549.yaml")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5 # confidence threshold
cfg.MODEL.WEIGHTS = "../../outputs/models/best_model.pth"
cfg.MODEL.DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

predictor = DefaultPredictor(cfg)

img = get_masks_no_norm("../temp_data/full_inspection/img1.png")
outputs = predictor(img)
instances = outputs["instances"].to("cpu")
boxes = instances.pred_boxes.tensor.numpy()
boxes_dict[name] = boxes
```

This will give bonding boxes for an image that are accurate and line up well with the prediction visulisations from training and testing.

However when using an onnx model created using the commands from detectron2s documentation:

`python3 ./detectron2/tools/deploy/export_model.py --config "../../outputs/models/config_custom_2025-09-12 17:13:50.286475.yaml" --output ./ --export-method tracing --sample-image ../datasets/dataset1/test.png --format onnx MODEL.WEIGHTS ../../outputs/models/best_model.pth MODEL.DEVICE cuda`

I get an onnx output but it performs differently to the original model.

I am then running this model with:

```python
session = ort.InferenceSession(model_path)

ai_image = image_pre_process()

# Run inference
inp_dict = {"x.1": ai_image}
outputs = session.run(None, inp_dict)
```

Ive given an example of how these detection are different below the object we are detecting are small so even being 2-3 pixels out could lead to missing the thing we want to find.

Detecron2 boxes:
```
[277.0877, 222.06036, 291.78217, 248.23471
[253.8955, 175.23312. 271.17606. 197.9112 ]
```

Onnx boxes:
```
[274.79288, 219.97232, 289.00266, 246.60608]
[250.75594, 174.43523, 268.79462, 197.45988]
```

Is there any reason you can think of why there would be a difference in the outputs from the Onnx model and the original detectron2 model?

Thank you for any help

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.