tensorflow / tensorflow/probability

Flipout Monte Carlo estimator

Open
#213 20 comments 1 reaction 1 assignee View on GitHub

@jvdillon is already working on this.

Since Nov 6, 2018.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

Hi all,

In the bayesian_neural_network.py example, how many samples are used by default to calculate the Flipout Monte Carlo estimator. If I refer to https://www.tensorflow.org/probability/api_docs/python/tfp/layers/Convolution2DFlipout:

It uses the Flipout gradient estimator to minimize the Kullback-Leibler divergence up to a constant, also known as the negative Evidence Lower Bound. It consists of the sum of two terms: the expected negative log-likelihood, which we approximate via Monte Carlo; and the KL divergence, which is added via regularizer terms which are arguments to the layer.

Monte Carlo approximation is used to approximate the expected negative log-likelihood but I wonder how many samples are used.
Is there a way to increase that number samples to better approximate my ELBO at each iteration?

Thanks

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.