google / google/uncertainty-baselines

Reproducing OOD scores for CIFAR-10 vs SVHN

Open
#286 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.6k
Forks
224
Avg merge
15h 36m
Merged PRs (30d)
2

Description

Hi there,

I'm currently trying to reproduce your results from the SNGP paper (https://arxiv.org/abs/2006.10108) and get a much higher AUCPR for the baseline deterministic WideResNet separating CIFAR-10 from SVHN as OOD set. I don't really see how the result for CIFAR-10 can also be worse than for CIFAR-100 since this relationship is reversed for all other tested methods.

Would be cool if you could have a look and advise if this is a typo in the paper or share how you arrive at the OOD scores for this setup. My results for the OOD task with a vanilla WRN are AUPR = 0.899 and AUROC = 0.931.
Screenshot 2021-01-26 at 08 28 51

Thanks a lot!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.