google-deepmind / google-deepmind/optax

LBFGS - Add functionality to skip updates

Open
#1,708 4 comments 0 reactions 0 assignees View on GitHub
type:feature
Dominant language
Python
Stars
2.3k
Forks
369
Avg merge
10h 15m
Merged PRs (30d)
7

Description

Would it be possible to add the ability to skip updates to the lbfgs algorithm, in case a certain curvature condition does not hold? You can find the relevant information on page 537 of the book "Numerical Optimization" by Nocedal and Wright.

While it isn't great to skip updates, it prevents the algorithm from breaking when the Wolfe conditions are not satisfied (sometimes, one needs to use backtracking line search due to computational restrictions).

This would mean to add about 20 lines to the existing optax code - I'm happy to provide my current implementation.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.