phadej / phadej/regression-simple

Make initial choice of lambda for Levenberg–Marquardt algorithm configurable

Open
#8 6 comments 0 reactions 1 assignee View on GitHub

@phadej is already working on this.

Since Jan 15, 2023.

Dominant language
Haskell
Stars
7
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Thanks for this tiny beautiful library! I've been looking for a long time for regressions without unreasonable dependency footprint.

The problem I have is that the initial guess of lambda0 in Levenberg–Marquardt algorithm is too high, like, 1e14, which makes delta too small, immediately triggering relDiff < 1e-10 condition. The workaround is to scale the function so that gradient is gentler, and lambda0 gets below 1e8 or similar:

    maxGrad = maximum $ 1 : map (\(x, _) -> x * log x) xys -- maxGrad is ~1e6
    scale = 1e8 / maxGrad / maxGrad -- scale is ~1e-4

    scaledFits = levenbergMarquardt1
      (\a (x, y) -> (y, mult scale (a * x * log x), mult scale (x * log x)))
      (mult (inv scale) initA)
      xys

    fit = scaleFit scale $ NE.last scaledFits

Could there be a way to specify the initial choice of lambda?

Alternatively, maybe if LM exits on the very first iteration, it might make sense to retry with a smaller lambda automatically.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.