Speculative dynamic HMC

Summary:

The dynamic HMC algorithm can be parallelized to run the forward and backward sweeps in independence. See for a description and a decent evaluation the thread on discourse:

https://discourse.mc-stan.org/t/parallel-dynamic-hmc-merits/10895

By using 2 cores we can be speed up things by 35%.

Description:

A prototype implementation is here:

https://github.com/stan-dev/stan/blob/parallel-nuts-2/src/stan/mcmc/hmc/nuts/base_nuts.hpp#L159

That version is based on the 2.20.0 sampler.

The branch is not runnable without a special setup of math due to the requirement of the TBB. A variant of the branch https://github.com/stan-dev/math/tree/parallel-ad-tape-3 can make this one run, but no attempt is made at the moment to give clear instructions.

This issue is here to document that we can get 35% speedup if we do a speculative HMC

Reproducible Steps:

See discourse for all of this.

Current Output:

Expected Output:

Faster samples.

Additional Information:

Provide any additional information here.

Current Version:

v2.20.0

Read stan/mcmc/hmc/nuts/base_nuts.hpp around line 159 and compare the parallel-nuts-2 prototype with the v2.20.0 sampler. Review the linked math branch and Discourse thread first, since the issue says the prototype needs a special TBB setup. Done means a runnable version that demonstrates the reported 35% speedup.