No detection when converting detectron2 model that use Lazy Configs
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
I successfuly exported and converted Detectron2 Mask RCNN R50-FPN ONNX into TensorRT and built an engine but it doesn't detect or very very rarely detects anything, even if it does it's so far below, barely above base NMS (~0.254) score.
Used 1344x1344 input image tensor
I tried exporting ONNX with opset 11 12 15 16 20 they all same result.
ONNXRuntime detects just fine and similar mAP but slower inference time by ~20%.
Environment
TensorRT Version:
TensorRT-10.9.0.34
NVIDIA GPU:
T1000 Mobile
NVIDIA Driver Version:
572.60
CUDA Version:
12.6
CUDNN Version:
9.6.0
Operating System:
Windows 11 24H2 26100.3775
Python Version (if applicable):
3.11
PyTorch Version (if applicable):
2.6.0+cu12
Relevant Files
changed part of create_onnx.py
# Set up Detectron 2 model configuration.
# lazy config
with open(config_file, 'rb') as f:
self.det2_cfg = cloudpickle.load(f)
self.det2_model = instantiate(self.det2_cfg.model)
self.det2_model.to(self.det2_cfg.train.device)
checkpointer = DetectionCheckpointer(self.det2_model)
checkpointer.load(weights)
self.det2_model.eval()
self.fpn_out_channels = self.det2_cfg.model.backbone.out_channels
self.num_classes = self.det2_cfg.model.roi_heads.num_classes
self.first_NMS_max_proposals = self.det2_model.proposal_generator.post_nms_topk[False] # is train = False
self.first_NMS_iou_threshold = self.det2_cfg.model.proposal_generator.nms_thresh
self.first_NMS_score_threshold = 0.01 # 1%
self.first_ROIAlign_pooled_size = self.det2_cfg.model.roi_heads.box_pooler.output_size
self.first_ROIAlign_sampling_ratio = self.det2_cfg.model.roi_heads.box_pooler.sampling_ratio
self.first_ROIAlign_type = self.det2_cfg.model.roi_heads.box_pooler.pooler_type
self.second_NMS_max_proposals = self.det2_model.roi_heads.box_predictor.test_topk_per_image if max_det is None else max_det
self.second_NMS_iou_threshold = self.det2_model.roi_heads.box_predictor.test_nms_thresh
self.second_NMS_score_threshold = self.det2_cfg.model.roi_heads.box_predictor.test_score_thresh
self.second_ROIAlign_pooled_size = self.det2_cfg.model.roi_heads.mask_pooler.output_size
self.second_ROIAlign_sampling_ratio = self.det2_cfg.model.roi_heads.mask_pooler.sampling_ratio
self.second_ROIAlign_type = self.det2_cfg.model.roi_heads.mask_pooler.pooler_type
self.mask_out_res = 28
config file is new_baselines/mask_rcnn_R_50_FPN_100ep_LSJ.py
Model link:
resulting converted onnx
Steps To Reproduce
detectron2 export -> ONNX -> convert for TensorRT -> build engine
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the changed Detectron2 lazy-config section of create_onnx.py and the referenced new_baselines/mask_rcnn_R_50_FPN_100ep_LSJ.py configuration. Reproduce the 1344x1344 export and compare ONNXRuntime with the TensorRT engine, focusing on the conversion and detection outputs. Done means TensorRT detections have comparable results to ONNXRuntime rather than near-zero scores.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- ai, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100