facebookresearch / facebookresearch/detectron2

Potential bug on drawing segmentation mask with holes

Open
#1,771 4 comments 4 reactions 0 assignees View on GitHub
bug contributions welcome
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

## Instructions To Reproduce the 🐛 Bug:
1. model has predicted segmentation masks with holes. (white area are predicted mask)

![mask](https://user-images.githubusercontent.com/39639112/87484601-975f5e80-c604-11ea-9fc8-0868ebc38b10.png)

2. When `Visualizer.draw_instance_predictions()` create `GenericMask` and call `Visualizer.overlay_instances`

3. `Visualizer.overlay_instances` draws mask with "polygon" formate, which considers the holes inside the prediction (which have it own polygon) as part of the prediction.

![Screen Shot 2020-07-14 at 7 04 55 PM](https://user-images.githubusercontent.com/39639112/87484744-fde47c80-c604-11ea-9fd6-b8b0523e93ab.png)

4. This would induce result like this, where the holes are also considered as predicted area.

![prediciton](https://user-images.githubusercontent.com/39639112/87484757-08067b00-c605-11ea-96f7-47413a7dd845.png)

5. This is mainly because when the class::GenericMask return the polygon, it does not consider the hole information

![Screen Shot 2020-07-14 at 7 09 46 PM](https://user-images.githubusercontent.com/39639112/87484967-9975ed00-c605-11ea-9e75-34858d6f0a25.png)

and this have to do with `cv2.findContours(mask.astype("uint8"), cv2.RETR_CCOMP, cv2.CHAIN_APPROX_NONE)` return holes inside the mask but there's not subsequrnce operation to filter out those polygon
![Screen Shot 2020-07-14 at 8 29 30 PM](https://user-images.githubusercontent.com/39639112/87489385-e14e4180-c610-11ea-9602-d885cc63342a.png)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.