google / google/flax

`serialization.from_state_dict` does not restore to jax.Arrays

Open
#3,999 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
7.3k
Forks
833
Avg merge
5h 11m
Merged PRs (30d)
5

Description

Deserialising a single jax.Array with `flax.serialization` does not reconstruct `jax.Array`correctly even when they are present in the target pytree.

See

```python
import jax
import jax.numpy as jnp
import flax
from flax import serialization

pt = {'a':jnp.ones((3,4))}

bdata = serialization.to_bytes(pt)
pt_loaded = serialization.from_bytes(pt, bdata)

print("Standard Deserialization output type:", type(pt_loaded['a']))
> Standard Deserialization output type:
```

This is because there is no rule for how to deal with jax.Arrays.

To fix it, one needs to register this rule
```python
serialization.register_serialization_state(
type(pt['a']),
lambda x: x,
lambda x, sd: jax.numpy.asarray(sd, dtype=x.dtype),
override=True
)
bdata = serialization.to_bytes(pt)
pt_loaded = serialization.from_bytes(pt, bdata)

print("type Deserialization output type:", type(pt_loaded['a']))
> type Deserialization output type:
```

Would it be possible to get this inside of flax itself?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.