huggingface / huggingface/pytorch-image-models
[FEATURE] Add the Distinction Maximization Loss (DisMax) to notably improve the OOD detection performance on ImageNet
- Dominant language
- Python
- Stars
- 37.1k
- Forks
- 5.2k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 34
Description
We suggest adding DisMax loss to improve OOD detection:
https://arxiv.org/abs/2205.05874
It improves the AUROC OOD detection performance of a ResNet18 trained on ImageNet by almost 25% using the hard ImageNet-O as out-of-distribution.
All it is required is to replace the SoftMax loss (i.e., the combination of the linear output layer, the SoftMax activation, and the cross-entropy loss) with the DisMax loss (see details how it is done in the code below).
The code is basically ready. It is essentially a matter of integrating into this lib:
https://github.com/dlmacedo/distinction-maximization-loss/
Using DisMax without FPR, which resulted in the above-mentioned result, no hyperparameter is required to be tuned.
Contributor guide
Research direction
Read the linked DisMax paper and the referenced distinction-maximization-loss implementation first, then locate this repository's existing SoftMax and cross-entropy training path. Integrate DisMax as the proposed replacement and verify the expected OOD detection improvement on ImageNet with ImageNet-O, using AUROC as the stated success measure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100