huggingface / huggingface/pytorch-image-models

[FEATURE] Add the Distinction Maximization Loss (DisMax) to notably improve the OOD detection performance on ImageNet

Open
#1,382 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
37.1k
Forks
5.2k
Avg merge
1d 13h
Merged PRs (30d)
34

Description

We suggest adding DisMax loss to improve OOD detection:

https://arxiv.org/abs/2205.05874

It improves the AUROC OOD detection performance of a ResNet18 trained on ImageNet by almost 25% using the hard ImageNet-O as out-of-distribution.

All it is required is to replace the SoftMax loss (i.e., the combination of the linear output layer, the SoftMax activation, and the cross-entropy loss) with the DisMax loss (see details how it is done in the code below).

The code is basically ready. It is essentially a matter of integrating into this lib:

https://github.com/dlmacedo/distinction-maximization-loss/

Using DisMax without FPR, which resulted in the above-mentioned result, no hyperparameter is required to be tuned.

Contributor guide

Open the contributing guide

Research direction

Read the linked DisMax paper and the referenced distinction-maximization-loss implementation first, then locate this repository's existing SoftMax and cross-entropy training path. Integrate DisMax as the proposed replacement and verify the expected OOD detection improvement on ImageNet with ImageNet-O, using AUROC as the stated success measure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.