greta-dev / greta-dev/greta

automatically decentre symmetric distributions

Open
#47 6 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
607
Forks
67
Avg merge
3d 8h
Merged PRs (30d)
1

Description

In some hierarchical models, the usual 'centred' formulation of a distribution can cause convergence problems due to a strongly [funnel-shaped posterior](http://twiecki.github.io/blog/2017/02/08/bayesian-hierchical-non-centered). A solution to this is to explicitly use an equivalent non-centred version of the hierarchical model.

centred:
```r
mu = normal(0, 1)
sigma = lognormal(0, 1)
x = normal(mu, sigma)
```

non-centred:
```r
mu = normal(0, 1)
sigma = lognormal(0, 1)
x_raw = normal(0, 1)
x = mu + sigma * x_raw
```

Because greta defines an unconstrained node for each parameter anyway, it should be possible to **automatically implement the non-centred version under the hood**. I.e. to define the following Tensors in the TF graph:

```r
# define the unconstrained node
xnode_free <- tf$Variable(...)

# create an uncentred version (like x_raw above)
xnode_uncentred <- tf_from_free(xnode_free)

# centre it; applying the mean and sd tensorflow values it depends on
xnode <- centre(xnode_uncentred)
```

The density could then still be defined on the centred version, or a few FLOPs might be saved by defining a reduced version of the density and applying it to `xnode_uncentred`. E.g. for the normal distribution, just: `exp(-0.5 * x ^ 2) * 0.3989 / sigma`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.