facebookresearch / facebookresearch/detectron2

Angle-based matching for RRPN

Open
#4,872 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

## 🚀 Feature

Add a parameter to account for the difference of angles between the proposed region and the ground truth in the anchor matcher when using RRPN.

## Motivation & Examples

The proposed feature could improve the quality of the proposed regions generated by RRPN.
In the original paper “Arbitrary-Oriented Scene Text Detection via Rotation Proposals” (https://arxiv.org/abs/1703.01086), during training, regions are selected as positive R-anchors if they have (i) the highest IoU overlap or an IoU larger than 0.7 with respect to the ground truth, and (ii) an intersection angle with respect to the ground truth of less than π/12. Negative R-anchors are characterized by an IoU lower than 0.3 or an IoU larger than 0.7 but with an intersection angle with a ground truth larger than π/12.

To my understainding, the current implementation of detectron2 does not use the second criteria, which can result in predicted bounding boxes with large angle differences compared to the ground truth, especially in the context of object detection in natural images, when the object rotated bbox have a large variation of angles. To address this issue, I suggest adding a parameter angle_threshold to the anchor_matcher function used in `detectron2/modeling/rpn/rrpn.py ` and modifying its logic to take into account the angle difference between anchors and targets. The angle_threshold parameter would specify the maximum allowed angle difference between the anchor and ground truth bounding boxes.

Here is an example implementation (just a starting point) of the modified `Matcher ` class (defined in `detectron2/modeling/matcher.py`):
```{python}

class Matcher(object):
def __init__(
self,
thresholds: List[float],
labels: List[int],
allow_low_quality_matches: bool = False,
angle_threshold: float = 0.0 # new parameter
):

def __call__(
self, match_quality_matrix: torch.Tensor, angle_diff_matrix: torch.Tensor
) -> Tuple[torch.Tensor, torch.Tensor]:

# change the logic of the implementation to take into account also the angle_threshold and angle_diff_matrix.
# angle_diff_matrix and angle_threshold can be optional parameters.

```

## Note

This feature would be relevant to many users who are interested in rotated object detection.

In my experince training a rotated faster-rcnn on a private dataset, predicted bbox with a large angle difference compared to ground trouth are a problem. However, I was not able to find much empirical evidence on the usefulness of this feature beyond the RRPN paper. Therefore, it would be useful to get feedback from other users and researchers on the potential benefits and drawbacks of implementing this feature.

Reference: Ma, Jianqi, et al. "Arbitrary-Oriented Scene Text Detection via Rotation Proposals." IEEE Transactions on Multimedia 20.11 (2018): 3111-3122.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.