determinant algorithm

Description

I was benchmarking determinant and log_determinant today.

For reverse mode for each of these functions you need to evaluate the determinant, and to get the gradients you need the inverse (it's actually the inverse -- no solves to be done -- here's a link to the math).

For the current implementation of determinant(m), we use m.determinant() and m.inverse() separately in the Eigen code.

For the current implementation of log determinant(m), we do m_piv = m.fullPivHouseholderQr() first and then m_piv.determinant() and m_piv.inverse().

The second is slower than the first I think because fullPivHouseholderQr is so conservative.

partialPivotLU however is faster than both methods. The limitation in the Eigen docs is it's only stable it if the inverse exists (https://eigen.tuxfamily.org/dox/classEigen_1_1PartialPivLU.html).

Anyone see any problem with switching to partialPivotLU for both since if the inverse doesn't exist our reverse mode autodiff fails anyway?

@bgoodri

Current Version:

v3.3.0

Start by locating the determinant(m) and log determinant(m) implementations in the Eigen code and compare their current determinant, inverse, and fullPivHouseholderQr calls with partialPivotLU. Benchmark the alternatives and verify that reverse-mode autodiff still behaves correctly when the matrix is invertible; done means a justified, tested implementation choice for both functions.