google-deepmind / google-deepmind/optax

Add __hash__ function to avoid unnecessary recompilations

Open
#353 5 comments 1 reaction 0 assignees View on GitHub
type:feature
Dominant language
Python
Stars
2.3k
Forks
369
Avg merge
10h 15m
Merged PRs (30d)
7

Description

Example:

```python
from functools import partial
import jax
import jax.numpy as jnp
import optax

@partial(jax.jit, static_argnames=("optimizer",))
def train_step(optimizer, opt_state):
print("compiling")

def train():
optimizer = optax.adam(1e-2)
opt_state = optimizer.init({"foo": jnp.zeros((100, 100))})

for _ in range(10):
train_step(optimizer, opt_state)

train()
train()
train()
```

This prints:

```
compiling
compiling
compiling
```

Even though the same optimizer is used every time. This could be avoided by supplying meaningful `__hash__` attributes based on the optimizer parameters, or memoizing the wrappers in `alias.py`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.