patrick-kidger / patrick-kidger/optimistix
Using `optimistix` with an `equinox` model
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 623
- Forks
- 54
- PR merge metrics
- No merged PRs in 30d
Description
Hi everyone, thanks for the great library and apologies in advance for this basic question.
I'm trying to find the true minimum of a small neural network, and I thought of using a solver from optimistix together with an equinox model. However, I haven't been able to make the two work together.
Here is a minimal snippet which fails:
import jax
import jax.numpy as jnp
import equinox as eqx
import optimistix as optx
jax.config.update("jax_enable_x64", True)
X = jax.random.normal(jax.random.PRNGKey(0), (2000, 8))
@jax.vmap
def function(x):
return x[0] + x[1]**2 + jnp.cos(x[2]) + jnp.sin(x[3]) + x[4]*x[5] + (x[6]*x[7])**3
y = function(X).reshape(-1, 1)
model = eqx.nn.MLP(in_size=8, out_size=1, width_size=4, depth=2, activation=jax.nn.silu, key=jax.random.PRNGKey(0))
static, params = eqx.partition(model, eqx.is_inexact_array)
def loss_fn(params, static, X, y):
model = eqx.combine(params, static)
return jnp.sum((jax.vmap(model)(X) - y)**2)
solver = optx.Newton(rtol=1e-5, atol=1e-5)
sol = optx.minimise(loss_fn, solver, params)
I'm getting TypeError: Cannot determine dtype of <PjitFunction of <function silu at 0x742fde959300>>.
What am I doing wrong?
Thank you in advance.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the minimal reproduction and tracing the interaction between eqx.partition/combine, loss_fn, and optx.minimise, with attention to the reported dtype error. No repository files or tests are named; done means establishing whether this Equinox model and Optimistix solver combination is supported and identifying the required project-level resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100