greta-dev / greta-dev/greta

increase default number of chains... by a lot

Open
#294 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
607
Forks
67
Avg merge
3d 8h
Merged PRs (30d)
1

Description

I have been massively underestimating the power of tensorflow probability's tensor-fied MCMC algorithms (that greta uses). There is a huge efficiency gain to be had from setting the number of chains to a large number in greta.

Some super-rough efficiency (effective samples per second) estimates for the linear regression example on the greta website homepage, leaving all other HMC parameters at the defaults:

chains | time | avg eff. samples | efficiency
-|-|-|-
4 | 18s | 3,330 | 185
8 | 19s | 5,605 | 295
16 | 21s | 10,000 | 477
32 | 21s | 24,360 | 1,160
64 | 25s | 53,100 | 2,124
256 | 36s| 160,236 | 4,451
1024 | 66s | 856,680 | 12,980

This is due to two things:
- tensorflow ops are much more efficient if they are large, and this approach computes values for multiple chains by adding another dimension of size `chains` to all operations
- there is much more information with which to tune the sampler during warmup, as the chains share tuning information and parameters

So IMO this approach should be promoted much more, and the default number of chains in `mcmc()` should probably be increased. That should have the dual effect of improving the efficiency of the default sampler, and also making people aware that increasing the number of chains is a good idea.

`chains = 32` seems a reasonable number of chains based on these estimates. With more than that there become too many samples to summarise and plot efficiently.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.