Proximal gradient and L-BFGS

Hi! Thank you for making Optimistix!

We're currently relying on JAXopt in NeMoS, but we're looking to transition to Optimistix (+ Optax).
For this, we would need implementations of proximal gradient descent and L-BFGS. We saw that there are plans to implement L-BFGS and some discussion about constrained optimization.
Are these currently in development by any chance? In case you're interested, we'd be happy to contribute.
We're still getting familiar with the libraries and would appreciate guidance on how best to approach this. Some initial thoughts:

• Proximal gradient: I made a toy implementation for Lasso and Ridge that chains optax.sgd and a step imitating Optax's projections which seems to work with optimistix.minimise. Do you think this makes sense, or should the proximal operator be handled inside Optimistix instead?
• L-BFGS: In my experiments, wrapping optax.lbfgs with optimistix.OptaxMinimiser runs without Optax’s line search but fails when it’s enabled.
  • Would it make sense and be feasible to turn OptaxMinimiser into a descent object, then combine it with Optimistix’s searches?
  • Maybe modify the wrapper to support Optax’s line search transformations?
  • Or drop Optax and write the required descent step in Optimistix?

@BalzaniEdoardo, @billbrod, @sjvenditto, @gviejo

Start by reviewing the linked plans for L-BFGS and constrained optimization, then inspect Optimistix’s minimise entry point and OptaxMinimiser behavior with optax.sgd and optax.lbfgs. Clarify the intended proximal-operator and line-search boundaries with maintainers; done should include agreed implementations for both methods and tests covering their stated behavior.