Why does LSTMCell keep rngs in its state?
- Dominant language
- Jupyter Notebook
- Stars
- 7.3k
- Forks
- 833
- Avg merge
- 5h 11m
- Merged PRs (30d)
- 5
Description
It seems that LSTMCell keeps rngs in its state:
https://github.com/google/flax/blob/a8a192ff167f8b25b9b568cfece44ef043a82dad/flax/nnx/nn/recurrent.py#L137
Is this intentional? Why?
I stumbled upon this because my recipe for checkpointing breaks when my model contains an LSTM:
```
import orbax.checkpoint as ocp
def savemodel(model, path):
_, state = nnx.split(model)
checkpointer = ocp.StandardCheckpointer()
checkpointer.save(path, state)
```
Calling `savemodel(model, path)` throws:
```
TypeError: JAX array with PRNGKey dtype cannot be converted to a NumPy array. Use jax.random.key_data(arr) if you wish to extract the underlying integer array.
```
This was surprising because I've been using that recipe before and never had a problem while using other non-LSTM modules.
Contributor guide
Assessment
This issue has not been assessed yet.