tensorflow / tensorflow/probability
Flipout Monte Carlo estimator
@jvdillon is already working on this.
Since Nov 6, 2018.
- Dominant language
- Jupyter Notebook
- Stars
- 4.4k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
Hi all,
In the bayesian_neural_network.py example, how many samples are used by default to calculate the Flipout Monte Carlo estimator. If I refer to https://www.tensorflow.org/probability/api_docs/python/tfp/layers/Convolution2DFlipout:
It uses the Flipout gradient estimator to minimize the Kullback-Leibler divergence up to a constant, also known as the negative Evidence Lower Bound. It consists of the sum of two terms: the expected negative log-likelihood, which we approximate via Monte Carlo; and the KL divergence, which is added via regularizer terms which are arguments to the layer.
Monte Carlo approximation is used to approximate the expected negative log-likelihood but I wonder how many samples are used.
Is there a way to increase that number samples to better approximate my ELBO at each iteration?
Thanks
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.