Various comments

Open
#1 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Quiet
Tech stack
julia
Domain
tooling

Research direction

Start by reviewing the current LineSearch.jl API and comparing the questions raised about Hager-Zhang in NonlinearSolve.jl, inner products in Optim.jl, and manifold step limits in Manopt.jl. No file or test is named, so the scope must first be narrowed to a specific proposal; done would require an agreed design, implementation target, and corresponding validation.

Written by the indexing model from the issue text.

Description

question

A few comments about line searches, potentially relevant to the work here.

  1. Hager-Zhang line search is in my experience a very good method for gradient-based line searches but it seems to be absent from NonlinearSolve.jl , other than through LineSearchesJL. Is it planned to be re-implemented here?
  2. Nonlinear gradient-based line searches actually need to compute inner products between gradient at a point and vector-transported descent direction. See here for example: https://arxiv.org/pdf/2207.01855.pdf . It means that for constrained optimization we need to do the transport (projection would often be fine but not necessarily always the best choice).
  3. The inner product actually doesn't require the full gradient. Optim.jl currently doesn't exploit it but it's something that crossed my mind when analyzing the code. We should sometimes be able to compute such products faster than the full gradient + dot. For example forward mode AD would handle this perfectly fine. Maybe that's something you could consider.
  4. Default max stepsize: I've encountered some problems in which the stepsize found by line search greatly exceeded (by orders of magnitude) the injectivity radius of exponential map. Manopt.jl patches it by using a manifold-dependent max stepsize (for example $\pi$ for a sphere). LineSearches.jl always defaults to Inf.
Dominant language
Julia
Stars
5
Forks
7
Avg merge
1h 51m
Merged PRs (30d)
12

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from SciML/LineSearch.jl

All issues in SciML/LineSearch.jl

Similar issues

More Julia issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.