[FR] low rank HMC?

Summary:

It would be nice to have versions of HMC/NUTS that support a low-rank plus diagonal metric a la L-BFGS.

This will require the following.

1. low_rank_e_metric and low_rank_e_point like dense_e_metric.hpp and dense_e_point.hpp which would use the code from pathfinder for getting the inverse hessian from here

2. low_rank_e_nuts and adapt_low_rank_e_nuts class like diag_e_nuts.hpp and adapt_diag_e_nuts.hpp

There's more information about this in @bbbales2 thesis, and refined in a joint arXiv paper with @pourzanj and @avehtari:

• Selecting the Metric in Hamiltonian Monte Carlo.

Current Version:

v2.29.0

Start by reading dense_e_metric.hpp, dense_e_point.hpp, diag_e_nuts.hpp, and adapt_diag_e_nuts.hpp to understand the existing metric and NUTS class structure. Then inspect the inverse-Hessian code linked in services/pathfinder/single.hpp and the referenced paper; done means implementing the requested low-rank metric and NUTS variants with adaptation.