google / google/edward2

Update the design of weight sampling in the BNN layers.

Open
#174 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
712
Forks
77
Avg merge
9h 39m
Merged PRs (30d)
1

Description

TFP has been updated such that `tfp.distributions.*` objects can be initialized in a tape-safe manner. I.e., it's now possible to create a distribution within one tape, and use it in another tape. [This test](https://github.com/google/edward2/blob/master/edward2/tensorflow/layers/dense_test.py#L151) now would no longer fail even if we didn't [create new weight RVs on each call](https://github.com/google/edward2/blob/master/edward2/tensorflow/layers/dense.py#L84).

However, we still need to be able to sample new values for the weights (also noted in https://github.com/google/edward2/commit/eb4f33c4eed9c0375623d0fd3add9c2d83b559ad), and be able to override this with tracers to enforce, for example, the use of the mean.

One option is to check if the layer weights are ed.RandomVariables, and if so, call `.sample()` within the layer (and regularizers) as needed. Tracers could be updated to override `.sample()` to return the mean if desired.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.