facebookresearch / facebookresearch/detectron2

Multi-label Detection

Open
#3,519 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

## 🚀 Feature
I checked the repo and some issues, saw that there is no multi-label support currently for detectron2 models. It is quite relevant topic when real-world datasets are considered. Do you consider adding multi-label solution, or can you show a way about how to implement it?

## Motivation & Examples
For my case, I am using DriveU Traffic Light Dataset and trying to detect traffic lights according to their aspects (bulb numbers) and also directions. There are also several pictograms that could be considered (like pedestrian light, bus light, tram light etc.)
Another case might be a dataset contains animals and also breeds of them so there is a sort of hierarchical relationship in classes.
One can consider vehicles and their brands which is again a multi-label situation.
There are many such cases that are multi-label in daily life.

For my own case, since I have 2 different and independent specifications (bulb number and direction), I was considering having 2 heads each are responsible for only 1 group of classes, and summing their losses.
Another approach might be using BCELoss + sigmoid.
However, I am not sure how to load the dataset and labels. For instance, can I label the same bounding box with 2 classes? (like writing class: 1 bbox1 and also class: 5 bbox1) or can I put as class: [0,0,1,1,0,0...] bbox: ... ? I think for BCELoss + sigmoid approach, I need class: [0,0,1,1,0,0...] kind of labelling.

I am also open to any helps. Thank you.

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. Start by reviewing Detectron2's model and dataset-labeling interfaces, then compare the proposed multi-head and BCELoss approaches. The scope and acceptance criteria remain unresolved, so a completed implementation is not yet defined.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.