facebookresearch / facebookresearch/detectron2
CustomPredictor Class Implementation, slightly different result.
- Dominant language
- Python
- Stars
- 34.7k
- Forks
- 7.9k
- PR merge metrics
- No merged PRs in 30d
Description
I have implemented a custom predictor class for MaskRCNN but every time I run inference I get slightly different bbox coordinates
With DefaultPredictor, I get
**_[395.03793, 308.10822, 472.1958, 526.22107]_**
But with this CustomPredictor I get
**_[393.92926, 307.31274, 473.9342, 522.9744]_**
Is there anything I'm missing in the below snippet? Any help would be greatly appreciated.
```
class CustomPredictor(nn.Module):
def __init__(self, model):
super(CustomPredictor, self).__init__()
self.preprocess_image = model.preprocess_image
self.backbone = model.backbone
self.proposal_generator = model.proposal_generator #Generates object proposals from the extracted features
self.box_pool = model.roi_heads.box_pooler #Pools features for each proposal
self.box_head = model.roi_heads.box_head #Computes features for each proposal
self.roi_heads = model.roi_heads #Filters out low-scoring proposals and performs instance segmentation on the remaining proposals
self.box_predictor = model.roi_heads.box_predictor
self.postprocess_image = model._postprocess
def forward(self, cv_image):
image_tensor = torch.from_numpy(cv_image.astype("float32").transpose(2, 0, 1))
height = image_tensor.shape[1]
width = image_tensor.shape[2]
batched_inputs = [{'image': image_tensor, 'height': height, 'width': width}]
with torch.no_grad():
images = self.preprocess_image(batched_inputs)
features = self.backbone(images.tensor)
proposals, _ = self.proposal_generator(images, features, None)
proposal_boxes = [x.proposal_boxes for x in proposals]
pred_instances, _ = self.roi_heads(images, features, proposals, None)
pred_instances = self.postprocess_image(pred_instances, batched_inputs, images.image_sizes)
#box_features_cpu = box_features.cpu().numpy()
pred_boxes = pred_instances[0]['instances'].pred_boxes
scores = pred_instances[0]['instances'].scores
pred_classes = pred_instances[0]['instances'].pred_classes
pred_masks = pred_instances[0]['instances'].pred_masks
# Create an instance of Instances class
pred_instances = Instances(
image_size=(height, width),
pred_boxes=pred_boxes,
scores=scores,
pred_classes=pred_classes,
pred_masks=pred_masks
)
return pred_instances
```
Contributor guide
Research direction
Start by comparing DefaultPredictor with the custom forward path, especially preprocess_image, backbone, proposal_generator, roi_heads, and _postprocess. Trace the inputs and image sizes through each entry point and establish whether the custom path produces matching boxes; the issue names no files or tests to run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100