increase default number of chains... by a lot
- Dominant language
- C++
- Stars
- 607
- Forks
- 67
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 1
Description
I have been massively underestimating the power of tensorflow probability's tensor-fied MCMC algorithms (that greta uses). There is a huge efficiency gain to be had from setting the number of chains to a large number in greta.
Some super-rough efficiency (effective samples per second) estimates for the linear regression example on the greta website homepage, leaving all other HMC parameters at the defaults:
chains | time | avg eff. samples | efficiency
-|-|-|-
4 | 18s | 3,330 | 185
8 | 19s | 5,605 | 295
16 | 21s | 10,000 | 477
32 | 21s | 24,360 | 1,160
64 | 25s | 53,100 | 2,124
256 | 36s| 160,236 | 4,451
1024 | 66s | 856,680 | 12,980
This is due to two things:
- tensorflow ops are much more efficient if they are large, and this approach computes values for multiple chains by adding another dimension of size `chains` to all operations
- there is much more information with which to tune the sampler during warmup, as the chains share tuning information and parameters
So IMO this approach should be promoted much more, and the default number of chains in `mcmc()` should probably be increased. That should have the dual effect of improving the efficiency of the default sampler, and also making people aware that increasing the number of chains is a good idea.
`chains = 32` seems a reasonable number of chains based on these estimates. With more than that there become too many samples to summarise and plot efficiently.
Contributor guide
Assessment
This issue has not been assessed yet.