phadej / phadej/regression-simple
Make initial choice of lambda for Levenberg–Marquardt algorithm configurable
@phadej is already working on this.
Since Jan 15, 2023.
- Dominant language
- Haskell
- Stars
- 7
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Thanks for this tiny beautiful library! I've been looking for a long time for regressions without unreasonable dependency footprint.
The problem I have is that the initial guess of lambda0 in Levenberg–Marquardt algorithm is too high, like, 1e14, which makes delta too small, immediately triggering relDiff < 1e-10 condition. The workaround is to scale the function so that gradient is gentler, and lambda0 gets below 1e8 or similar:
maxGrad = maximum $ 1 : map (\(x, _) -> x * log x) xys -- maxGrad is ~1e6
scale = 1e8 / maxGrad / maxGrad -- scale is ~1e-4
scaledFits = levenbergMarquardt1
(\a (x, y) -> (y, mult scale (a * x * log x), mult scale (x * log x)))
(mult (inv scale) initA)
xys
fit = scaleFit scale $ NE.last scaledFits
Could there be a way to specify the initial choice of lambda?
Alternatively, maybe if LM exits on the very first iteration, it might make sense to retry with a smaller lambda automatically.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.