google / google/uncertainty-baselines
Reproducing OOD scores for CIFAR-10 vs SVHN
Open
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 224
- Avg merge
- 15h 36m
- Merged PRs (30d)
- 2
Description
Hi there,
I'm currently trying to reproduce your results from the SNGP paper (https://arxiv.org/abs/2006.10108) and get a much higher AUCPR for the baseline deterministic WideResNet separating CIFAR-10 from SVHN as OOD set. I don't really see how the result for CIFAR-10 can also be worse than for CIFAR-100 since this relationship is reversed for all other tested methods.
Would be cool if you could have a look and advise if this is a typo in the paper or share how you arrive at the OOD scores for this setup. My results for the OOD task with a vanilla WRN are AUPR = 0.899 and AUROC = 0.931.

Thanks a lot!
Contributor guide
Assessment
This issue has not been assessed yet.