[Feature Request] Levenberg Marquardt
- Dominant language
- Python
- Stars
- 617
- Forks
- 57
- PR merge metrics
- No merged PRs in 30d
Description
# Feature
Implementation of Levenberg Marquardt optimizer.
# Motivation
In MATLAB, the `feedforwardnet` supports many optimizers, and some are not implemented in PyTorch, such as `trainlm` ([Levenberg Marquardt](https://www.mathworks.com/help/deeplearning/ref/trainlm.html)). From tests from our gourp and others, Levenberg Marquardt performs much better and needs less epochs to coverage than first-order optimizers (Adam, SGD with Momentum) on regression model. Some issues in PyTorch issue list hope for a implementation of Levenverg Marquardt algorithm, but there are no more reponse. It will be helpful to implement the Levenberg Marquardt optimizer and I am willing to help test.
# Addition
Here are some helpful information and codes.
- An introduction on Wikipedia. [Levenberg–Marquardt algorithm](https://en.m.wikipedia.org/wiki/Levenberg–Marquardt_algorithm)
- An implementation for Tensorflow. [tf-levenberg-marquardt](https://github.com/fabiodimarco/tf-levenberg-marquardt)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.