facebookresearch / facebookresearch/detectron2

Instance-Segmentation Predictions to COCO JSON Format

Open
#5,444 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

## 🚀 Feature
Taking the detailed output of instance segmentation, which provides masks that highlight individual objects, we transform this into a COCO JSON format. This process involves examining the spatial relationships between the segmented objects within each image. The resulting COCO JSON data is offering a structured representation of the predictions, which can be used as an BASE input for further annotations/labellings

## Motivation & Examples

- Lesion Detection
- Urban Planning and Monitoring
- Organ Segmentation
- Disaster Management
- Many More

in the above mentioned uses of image segmentation and deep learning, where the training dataset contains many images, where each image contains many multiple objects of similar or different classes, which requires manually annotating all the images

Pros
- More accurate annotations

Cons
- Time consuming
- Prone To Human Error
- Reduces Efficiency

but if a small subset of the same dataset is feed as the training dataset and a BASE model is trained, then you can use this feature or piece of code to convert the predictions made by the BASE model to annotations for the remaining images present in the dataset, which reduces the time taken significantly

Lets say you have 100 images, each containing 10-50 objects to be annotated, total objects an avg to be annotated is 3,000 objects.

now lets say you can annotated around 200-300 objects a day without affecting the accuracy and efficiency in the annotations, resulting in

- 10-12 days of only annotating the training dataset
- 3,000+ objects annotated

which can still contain human error

But if you train the BASE model with 10 images, and then perform predications on the remaining images with a minimum of 80% overall accuracy including 10% false positives and 10% false negatives, then on avg the changes to be made or new annotation to be marked per image will be reduced from 30 -> 6, therefore on avg resulting in

- 2 days of training the model ( 1-1.5 days of annotating the images and the rest for training the BASE model )
- 2 days for manually correcting the inaccuracies and missing data on the remaining images

on avg time taken is around 3-5days, when compared with manual annotations, this feature is 2-3 times ( 150-200% ) faster than the manual, traditional method

# Simple Example

### Input Image

![Image](https://github.com/user-attachments/assets/2e1ba8ef-055d-4d34-9a6c-29ef5839e9b8)

### Model

the pre-trained already available Instance-Segmentation Model ( COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml )

### Output Image

![Image](https://github.com/user-attachments/assets/3d34771b-2b07-4c2d-a4eb-ac67533c37b5)

these COCO annotations are made using above mentioned feature/piece of code

Contributor guide

Open the contributing guide

Research direction

No file, test, or entry point is named. Start by reviewing Detectron2's instance-segmentation prediction output and the COCO JSON annotation format. Done means the predicted masks and associated object information are converted into COCO JSON that can serve as input for further annotations.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.