facebookresearch / facebookresearch/detectron2
Angle-based matching for RRPN
- Dominant language
- Python
- Stars
- 34.7k
- Forks
- 7.9k
- PR merge metrics
- No merged PRs in 30d
Description
## 🚀 Feature
Add a parameter to account for the difference of angles between the proposed region and the ground truth in the anchor matcher when using RRPN.
## Motivation & Examples
The proposed feature could improve the quality of the proposed regions generated by RRPN.
In the original paper “Arbitrary-Oriented Scene Text Detection via Rotation Proposals” (https://arxiv.org/abs/1703.01086), during training, regions are selected as positive R-anchors if they have (i) the highest IoU overlap or an IoU larger than 0.7 with respect to the ground truth, and (ii) an intersection angle with respect to the ground truth of less than π/12. Negative R-anchors are characterized by an IoU lower than 0.3 or an IoU larger than 0.7 but with an intersection angle with a ground truth larger than π/12.
To my understainding, the current implementation of detectron2 does not use the second criteria, which can result in predicted bounding boxes with large angle differences compared to the ground truth, especially in the context of object detection in natural images, when the object rotated bbox have a large variation of angles. To address this issue, I suggest adding a parameter angle_threshold to the anchor_matcher function used in `detectron2/modeling/rpn/rrpn.py ` and modifying its logic to take into account the angle difference between anchors and targets. The angle_threshold parameter would specify the maximum allowed angle difference between the anchor and ground truth bounding boxes.
Here is an example implementation (just a starting point) of the modified `Matcher ` class (defined in `detectron2/modeling/matcher.py`):
```{python}
class Matcher(object):
def __init__(
self,
thresholds: List[float],
labels: List[int],
allow_low_quality_matches: bool = False,
angle_threshold: float = 0.0 # new parameter
):
def __call__(
self, match_quality_matrix: torch.Tensor, angle_diff_matrix: torch.Tensor
) -> Tuple[torch.Tensor, torch.Tensor]:
# change the logic of the implementation to take into account also the angle_threshold and angle_diff_matrix.
# angle_diff_matrix and angle_threshold can be optional parameters.
```
## Note
This feature would be relevant to many users who are interested in rotated object detection.
In my experince training a rotated faster-rcnn on a private dataset, predicted bbox with a large angle difference compared to ground trouth are a problem. However, I was not able to find much empirical evidence on the usefulness of this feature beyond the RRPN paper. Therefore, it would be useful to get feedback from other users and researchers on the potential benefits and drawbacks of implementing this feature.
Reference: Ma, Jianqi, et al. "Arbitrary-Oriented Scene Text Detection via Rotation Proposals." IEEE Transactions on Multimedia 20.11 (2018): 3111-3122.
Contributor guide
Assessment
This issue has not been assessed yet.