google / google/flax

Add NNX support for legacy `jax.random.PRNGKey()`

Open
#4,231 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
7.3k
Forks
833
Avg merge
5h 11m
Merged PRs (30d)
5

Description

Currently, it doesn't seem possible to straightforwardly checkpoint (with Orbax) an NNX module that includes random keys (like with dropout), see https://github.com/google/orbax/issues/1105#issue-2488663946. This seems to be due to the new JAX random key type (https://jax.readthedocs.io/en/latest/jep/9263-typed-keys.html), which is used here (https://github.com/google/flax/blob/fc19c5dcc136b17d7a2c406831126d93bbdac6d7/flax/nnx/rnglib.py#L186) in NNX. Although Orbax has added individual support for the new type (see https://github.com/google/orbax/issues/620), saving `nnx.state(model)` that includes `dtype=key` doesn't seem to be possible.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.