tensorflow / tensorflow/probability
tfp.layers.DenseFlipout freezing standard deviations
Open
@jburnim is already working on this.
Since Jan 17, 2019.
layers
- Dominant language
- Jupyter Notebook
- Stars
- 4.4k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
Is it possible to only freeze the standard deviations in tfp.layers.DenseFlipout, tfp.layers.DenseLocalReparameterization, tfp.layers.DenseReparameterization during training?
They all have an argument for trainable however during training one might want to for example ensure that the standard deviations don't go below 1.0 and at which point we treat these layers as regular layers instead of BNN layers i.e. we only train the means, and not the standard deviations.
Therefore we don't want to switch trainable=False because we still want to train the means weights (and biases).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.