automatically decentre symmetric distributions
- Dominant language
- C++
- Stars
- 607
- Forks
- 67
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 1
Description
In some hierarchical models, the usual 'centred' formulation of a distribution can cause convergence problems due to a strongly [funnel-shaped posterior](http://twiecki.github.io/blog/2017/02/08/bayesian-hierchical-non-centered). A solution to this is to explicitly use an equivalent non-centred version of the hierarchical model.
centred:
```r
mu = normal(0, 1)
sigma = lognormal(0, 1)
x = normal(mu, sigma)
```
non-centred:
```r
mu = normal(0, 1)
sigma = lognormal(0, 1)
x_raw = normal(0, 1)
x = mu + sigma * x_raw
```
Because greta defines an unconstrained node for each parameter anyway, it should be possible to **automatically implement the non-centred version under the hood**. I.e. to define the following Tensors in the TF graph:
```r
# define the unconstrained node
xnode_free <- tf$Variable(...)
# create an uncentred version (like x_raw above)
xnode_uncentred <- tf_from_free(xnode_free)
# centre it; applying the mean and sd tensorflow values it depends on
xnode <- centre(xnode_uncentred)
```
The density could then still be defined on the centred version, or a few FLOPs might be saved by defining a reduced version of the density and applying it to `xnode_uncentred`. E.g. for the normal distribution, just: `exp(-0.5 * x ^ 2) * 0.3989 / sigma`.
Contributor guide
Assessment
This issue has not been assessed yet.