facebookresearch / facebookresearch/detectron2

Getting average precision class values from detectron2 model evaluation

Open
#5,101 8 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

## 🚀 Feature
A clear and concise description of the feature proposal.

When using the `inference_on_dataset` function from `detectron2.evaluation`, it only provides the overall mAP50 value, and does not provide class AP50 values. However, it does provide individual AP values. It would also be helpful if a similar feature could be added for precision and recall. If there is currently a method to do this, please let me know.

I spent a long time trying to create my own AP, Precision, and Recall evaluators which turned out to be very slow, difficult to use, and may not be completely precise. I hope others will not have to go through the trouble of making it from scratch.

If it were to be implemented, the precision, recall, and AP values with their corresponding classes, would be included as keys in the ordered dictionary generated from `inference_on_dataset`.

I am currently testing something with fastrcnn, but I do not know if the type of model I am using makes a difference for the `inference_on_dataset` function. To be clear, I am using the COCO format.

Contributor guide

Open the contributing guide

Research direction

Start with the inference_on_dataset function in detectron2.evaluation and review how COCO-format evaluation currently returns overall and individual AP values. Determine how class-level precision, recall, and AP50 should appear in its ordered dictionary, then verify that the requested values and corresponding classes are returned for the reported Faster R-CNN use case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.