google / google/uncertainty-baselines

Question about SNGP differences from the paper

Open
#640 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
1.6k
Forks
224
Avg merge
15h 36m
Merged PRs (30d)
2

Description

Hi. Thanks for keeping everything updated here. I noticed there are some differences between the SNGP implementation here and what is described in the paper which leaves a few questions I am curious about:

- The default settings here: https://github.com/google/uncertainty-baselines/blob/main/baselines/cifar/sngp.py#L155 place the ridge penalty at `1` instead of `1e-3` as mentioned in table 5 of the paper. Is this a result of using the Gaussian likelihood instead of the logistic?
- In conjunction with the point above, there seems to be an extra multiplication which is not included in the paper equations in the predictive variance (https://github.com/google/edward2/blob/main/edward2/tensorflow/layers/random_feature.py#L456) which multiplies by the ridge penalty again after inversion. Was this used in the original experiments? How is this justified?
- Regarding this comment: https://github.com/google/uncertainty-baselines/issues/258, it says that the current code used the Gaussian likelihood for simplification, but in the paper it seems to follow a one vs all logistic regression. Was a one-vs-all logistic regression used in the original training? or was it always a softmax even though the likelihood is logistic, or something else entirely?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.