facebookresearch / facebookresearch/detectron2
Multi-label Detection
- Dominant language
- Python
- Stars
- 34.7k
- Forks
- 7.9k
- PR merge metrics
- No merged PRs in 30d
Description
## 🚀 Feature
I checked the repo and some issues, saw that there is no multi-label support currently for detectron2 models. It is quite relevant topic when real-world datasets are considered. Do you consider adding multi-label solution, or can you show a way about how to implement it?
## Motivation & Examples
For my case, I am using DriveU Traffic Light Dataset and trying to detect traffic lights according to their aspects (bulb numbers) and also directions. There are also several pictograms that could be considered (like pedestrian light, bus light, tram light etc.)
Another case might be a dataset contains animals and also breeds of them so there is a sort of hierarchical relationship in classes.
One can consider vehicles and their brands which is again a multi-label situation.
There are many such cases that are multi-label in daily life.
For my own case, since I have 2 different and independent specifications (bulb number and direction), I was considering having 2 heads each are responsible for only 1 group of classes, and summing their losses.
Another approach might be using BCELoss + sigmoid.
However, I am not sure how to load the dataset and labels. For instance, can I label the same bounding box with 2 classes? (like writing class: 1 bbox1 and also class: 5 bbox1) or can I put as class: [0,0,1,1,0,0...] bbox: ... ? I think for BCELoss + sigmoid approach, I need class: [0,0,1,1,0,0...] kind of labelling.
I am also open to any helps. Thank you.
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by reviewing Detectron2's model and dataset-labeling interfaces, then compare the proposed multi-head and BCELoss approaches. The scope and acceptance criteria remain unresolved, so a completed implementation is not yet defined.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100