facebookresearch / facebookresearch/detectron2

CustomPredictor Class Implementation, slightly different result.

Open
#5,105 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

I have implemented a custom predictor class for MaskRCNN but every time I run inference I get slightly different bbox coordinates

With DefaultPredictor, I get
**_[395.03793, 308.10822, 472.1958, 526.22107]_**
But with this CustomPredictor I get
**_[393.92926, 307.31274, 473.9342, 522.9744]_**

Is there anything I'm missing in the below snippet? Any help would be greatly appreciated.

```
class CustomPredictor(nn.Module):
def __init__(self, model):
super(CustomPredictor, self).__init__()
self.preprocess_image = model.preprocess_image
self.backbone = model.backbone
self.proposal_generator = model.proposal_generator #Generates object proposals from the extracted features
self.box_pool = model.roi_heads.box_pooler #Pools features for each proposal
self.box_head = model.roi_heads.box_head #Computes features for each proposal
self.roi_heads = model.roi_heads #Filters out low-scoring proposals and performs instance segmentation on the remaining proposals
self.box_predictor = model.roi_heads.box_predictor
self.postprocess_image = model._postprocess

def forward(self, cv_image):
image_tensor = torch.from_numpy(cv_image.astype("float32").transpose(2, 0, 1))
height = image_tensor.shape[1]
width = image_tensor.shape[2]
batched_inputs = [{'image': image_tensor, 'height': height, 'width': width}]




with torch.no_grad():
images = self.preprocess_image(batched_inputs)

features = self.backbone(images.tensor)
proposals, _ = self.proposal_generator(images, features, None)


proposal_boxes = [x.proposal_boxes for x in proposals]



pred_instances, _ = self.roi_heads(images, features, proposals, None)
pred_instances = self.postprocess_image(pred_instances, batched_inputs, images.image_sizes)

#box_features_cpu = box_features.cpu().numpy()



pred_boxes = pred_instances[0]['instances'].pred_boxes
scores = pred_instances[0]['instances'].scores
pred_classes = pred_instances[0]['instances'].pred_classes
pred_masks = pred_instances[0]['instances'].pred_masks

# Create an instance of Instances class
pred_instances = Instances(
image_size=(height, width),
pred_boxes=pred_boxes,
scores=scores,
pred_classes=pred_classes,
pred_masks=pred_masks
)

return pred_instances
```

Contributor guide

Open the contributing guide

Research direction

Start by comparing DefaultPredictor with the custom forward path, especially preprocess_image, backbone, proposal_generator, roi_heads, and _postprocess. Trace the inputs and image sizes through each entry point and establish whether the custom path produces matching boxes; the issue names no files or tests to run.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.