facebookresearch / facebookresearch/detectron2
Getting average precision class values from detectron2 model evaluation
- Dominant language
- Python
- Stars
- 34.7k
- Forks
- 7.9k
- PR merge metrics
- No merged PRs in 30d
Description
## 🚀 Feature
A clear and concise description of the feature proposal.
When using the `inference_on_dataset` function from `detectron2.evaluation`, it only provides the overall mAP50 value, and does not provide class AP50 values. However, it does provide individual AP values. It would also be helpful if a similar feature could be added for precision and recall. If there is currently a method to do this, please let me know.
I spent a long time trying to create my own AP, Precision, and Recall evaluators which turned out to be very slow, difficult to use, and may not be completely precise. I hope others will not have to go through the trouble of making it from scratch.
If it were to be implemented, the precision, recall, and AP values with their corresponding classes, would be included as keys in the ordered dictionary generated from `inference_on_dataset`.
I am currently testing something with fastrcnn, but I do not know if the type of model I am using makes a difference for the `inference_on_dataset` function. To be clear, I am using the COCO format.
Contributor guide
Research direction
Start with the inference_on_dataset function in detectron2.evaluation and review how COCO-format evaluation currently returns overall and individual AP values. Determine how class-level precision, recall, and AP50 should appear in its ordered dictionary, then verify that the requested values and corresponding classes are returned for the reported Faster R-CNN use case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100