patrick-kidger / patrick-kidger/optimistix
Behavior of BFGS
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 623
- Forks
- 54
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I'm not sure if this is the desired behavior, but BFGS directly stops if f(y0) == 0. As an example :
from optimistix import minimise, BFGS, rms_norm
import equinox as eqx
import jax.numpy as jnp
@eqx.filter_jit
def f(y, args):
return jnp.sum(jnp.square(y))
N = 2
def f2(y, args):
return f(y, args) - N
y0 = jnp.ones((N,))
res = minimise(
f,
BFGS(rtol=1e-13, atol=1e-13, norm=rms_norm),
y0=y0,
max_steps=1024,
)
print(res.stats["num_steps"])
assert jnp.allclose(res.value, jnp.zeros((N,)))
res = minimise(
f2,
BFGS(rtol=1e-13, atol=1e-13, norm=rms_norm),
y0=y0,
max_steps=1024,
)
print(res.stats["num_steps"])
assert jnp.allclose(res.value, jnp.zeros((N,)))
This is because then the termination condition is already validated at first step.
It's not really a big problem since you just need to offset the cost by a constant but it took me a while to figure out where the problem was coming from.
Maybe the initial value for f_info should be set to infinity, or maybe this should be mentioned in the doc ?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the provided Python example first, then inspect the BFGS termination logic and initial f_info handling. Done means the behavior is resolved or documented, with both the f and f2 cases covered by a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100