fslaborg / fslaborg/FSharp.Stats

[Feature Request] Levenberg–Marquardt algorithm: Different increase and decrease for damping parameter λ

Open
#145 4 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
F#
Stars
227
Forks
58
Avg merge
55m
Merged PRs (30d)
1

Description

The current implementation of the [Levenberg–Marquardt algorithm](https://github.com/fslaborg/FSharp.Stats/blob/bf406eea4473d32aee966239eff0e6f39c00fb9a/src/FSharp.Stats/Fitting/NonLinearRegression.fs#L123) has the damping parameter (lambda) and and a factor which raises/lowers it (lambdaFactor) as parameters. For some problems it can be beneficial to have different values for the increase and decrease of the damping parameter.

> In this scheme, if a step is accepted, then λ is decreased by a fixed factor, say 10. If a step is rejected then λ is appropriately raised by a factor of 10. (...) We have much greater success using a factor of 2 or 3 on most problems. Additionally, we find that lowering λ by a larger factor than it is raised also produces more beneficial results. For many moderate sized problems decreasing by a factor of 3 and raising by a factor of 2 is adequate. For larger problems, decreasing by a factor of 5 and raising by a factor of 1.5 is better.
-- [Transtrum, Mark K; Sethna, James P (2012)](https://arxiv.org/abs/1201.5885)

That is why I'd like to propose to split the parameter "lambdaFactor" in "lambdaFactorIncrease" and "lambdaFactorDecrease". This change would break the function for everyone currently using it. My suggestion for this would be to rename the LevenbergMarquardt modules and abbreviate them with LM. The old module can then stay (marked as obsolete) and contain a version of the function, which simply takes one lambdaFactor and calls the new version of the function internally, inserting the one factor twice. This way those functions would still work and the new version would also be available with a shorter module name.

```
open System

module LM =
let estimateParams (...) lambdaInit lambdaFactorIncrease lambdaFactorDecrease =
(...)
let estimateParamsVerbose (...) lambdaInit lambdaFactorIncrease lambdaFactorDecrease =
(...)

[]
module LevenbergMarquardt =

[]
let estimateParams (...) lambdaInit lambdaFactor =
LM.estimateParams (...) lambdaInit lambdaFactor lambdaFactor

[]
let estimateParamsVerbose (...) lambdaInit lambdaFactor =
LM.estimateParamsVerbose (...) lambdaInit lambdaFactor lambdaFactor
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.