A plan for a more automated Optimization.jl
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 839
- Forks
- 100
- Avg merge
- 20h 43m
- Merged PRs (30d)
- 38
Description
Optimization.jl is okay, but it's not quite conforming to the standards of the SciML interface.
It conflaits the "solver package" with the "interface package". In some sense, the current
Optimization.jl is closer to DiffEqBase.jl than it is to DifferentialEquations.jl. The issue with
that is apparent when looking at the higher level interface. Acting as an interface base,
Optimization.jl does not take on the dependencies of any automatic differentiation package, nor
does it take on the dependencies of any solver package. Because of that, every time a user wants
to solve, they need to:
- Pick an automatic differentiation choice.
- Choose a solver.
But why? This isn't the point of the SciML solver conglomerate packages. This is the point of
the deeper interface part, for the hardcore users. We're missing the solve(prob) for
Optimization.jl, the "do it at least okay for me" that users know and love with our other packages.
This is very much needed since I am seeing on the Discourse users getting confused by the glutteny
of having so many optimization methods to choose and so little time to learn why a choice should
be made. Good error messages are now telling people "hey this solver doesn't support constraints"
and "this solver needs differentiation which you didn't define". But it doesn't go in the positive
direction, i.e. "here is a good enough solver for a problem that is differentiable with box constraints".
For Optimization.jl, most users would prefer that kind of hand holding.
The bigger question is, how do we get there? There's a few things to note:
- The PolyOpt() method derived from
sciml_trainis rather good as a decent default for cases which
are differentiable. sciml_train's automated AD choice mechanism can be revived here. Try/catch. Since the objective
function is called hundreds of times, sparing a few to test ADs is a perfectly fine trade-off, and
it can always be ignored by choosing an AD yourself of course.- If the problem isn't AD friendly, we should defult to a derivative-free local optimizer. Something
from NLopt has always done me well: I'd go toNLopt.LN_COBYLA. - We should have an
alg_hintfor:global(change to enums first!!!) that would then switch to
global methods. Default to multistart optimziation if differentiable, and otherwise default
to BBO. - How to do the swap is interesting. I think the simplest way to do this is to actually move the
current code to a subpackage OptimizationAD.jl and Optimization. - Those enums should be in SciMLBase. Why are they here?
- Should the AD supports be subpackages of their own to remove Requires?
- We need Enzyme
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review the current Optimization.jl structure and the proposed OptimizationAD.jl split, then trace PolyOpt(), sciml_train, alg_hint, and automatic-differentiation selection. Compare the proposal to the suggested SciMLBase enum changes and the mentioned NLopt and Enzyme integrations. Done means the default-selection architecture and package boundaries are agreed and implemented with coverage for differentiable, derivative-free, constrained, and global cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- api, developer-experience
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100