Potential Future Solvers/Algorithms

Nobody has claimed this yet.

I'm just going to keep a list here of solvers/algorithms that might be interesting to implement in SNLS. I'll keep adding to this list as I find more that might be of interest.

I cam across this one today: https://arxiv.org/abs/2112.02089 which could be seen as a variation of a Levenberg-Marquardt type solvers. I was just looking through it (alg 2 mainly), and it seems like it would fairly simple to implement given it's not too far off from what we already do. If the convergence properties hold for our type of problems this could be a very interesting one to use especially for our very stiff equations.

Another one that might be of interest would be potentially making use of something like the double-dogleg in our solvers. You can find a description of it at section 6.4 of https://doi.org/10.1137/1.9781611971200.ch6 or https://doi.org/10.1007/BF00932218 . Although, it seems that this is largely the same as the single dogleg with the caveat that earlier on it would be a bit more biased towards the newton step rather than the cauchy point. However, I'm not necessarily sure it would be worth adding unless it drastically seems to help our solvers, which I don't believe it will.

No file or test is named. Start by reading the existing SNLS solvers and then compare them with Algorithm 2 in the linked arXiv paper and the double-dogleg references; determine whether either approach is appropriate for SNLS's stiff equations and define convergence or solver-performance evidence before implementation.

Written by the indexing model from the issue text.