fslaborg / fslaborg/FSharp.Stats

Add more Optimization / Minimization algorithms

Open
#193 2 comments 2 reactions 0 assignees View on GitHub
enhancement priority-high up-for-grabs
Dominant language
F#
Stars
227
Forks
58
Avg merge
55m
Merged PRs (30d)
1

Description

**Is your feature request related to a problem? Please describe.**
Currently, there is no way to find the argument / function value that minimizes a target function. Such operations are often useful.

**Describe the solution you'd like**
I suggest implementing a relatively small set of algorithms that can handle a wide range of target functions. Specifically, I suggest implementing
* The [Nelder-Mead Method](https://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method), as it does not require any special properties of the target function or any derivative-information. Also, it is relatively simple.
* A very general solver for equality and inequality constrained problems. In particular, I suggest using [Sequential Quadratic Programming](https://en.wikipedia.org/wiki/Sequential_quadratic_programming).

This Sequential Quadratic Approach is also used in the default method chosen by the [scipy.optimize.minimize function](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html).
The scipy package is however relying on a Fortran implementation, and I'm not sure if F# native performance will be enough.
There is a big suite of [Test functions for optimization](https://en.wikipedia.org/wiki/Test_functions_for_optimization), and benchmarking could be done with a handful of these.

**Describe alternatives you've considered**
Use MathNET.Numerics instead, and write some glue code.
Also, it could be discussed if a different set of algorithms would make more sense. I'm not sure about the performance tradeoffs of using SQP as the de-facto standard for everything, that cannot be managed with Nelder-Mead.

**Additional context**
Add any other context or screenshots about the feature request here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.