Windowed HMC and CAR model instability

Hi All,
I've been testing out the new experimental windowed_adaptive_hmc function with a spatial conditional autoregressive model I'm working on. MRE is here with a comparison against a PyMC3/Theano implementation.

In the TFP version, I'm experiencing instability with the required Cholesky decomposition (InvalidArgumentError: Input matrix is not invertible.) . Although the model is very basic and needs improvement, this issue doesn't occur in the PyMC3 version and I get relatively sane inference.

I thought I'd open an issue, as this may well be related to the fixed num_adaptation_steps versus fixed total integration time argument. With the former strategy, bad choices of step size early on in the adaptation are likely to sling the leapfrog integrator into outer space, whereas all that happens with the latter strategy is that the number of steps increases and the algorithm slows.

Any thoughts?

Chris

Start by running the linked Colab MRE and inspecting the experimental windowed_adaptive_hmc path around the required Cholesky decomposition. Compare the TensorFlow Probability behavior with the PyMC3/Theano result and determine whether adaptation strategy is involved. Done means reproducing the instability, identifying its cause, and documenting or testing a reliable resolution.