google / google/flax

Regular dict representation not indented

Open
#3,280 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
7.3k
Forks
833
Avg merge
5h 11m
Merged PRs (30d)
5

Description

After the [dict migration](https://github.com/google/flax/pull/3193), Flax now returns regular dicts when calling the `.init`, `.init_with_output` and `.apply` Module methods. However the representation of regular dicts are not as readable compared to the indented version of FrozenDicts.

Regular dicts:
```
class MLP(nn.Module):
@nn.compact
def __call__(self, x):
x = nn.Dense(5)(x)
x = nn.relu(x)
return x
x = jnp.ones((1,3))
model = MLP()
params = model.init(jax.random.PRNGKey(0), x)['params']
state = TrainState.create(apply_fn=model.apply, params=params, tx=optax.adam(1e-3))
state
```
```
TrainState(step=0, apply_fn=, params={'Dense_0': {'kernel': Array([[ 0.37229332, -0.4265755 , -1.1151816 , -0.09558704, -0.62169886],
[-1.060781 , 1.0546707 , 0.33051118, -0.7090655 , 0.37682843],
[-0.30747807, -0.39064118, -0.25515485, 0.5127583 , -0.5559202 ]], dtype=float32), 'bias': Array([0., 0., 0., 0., 0.], dtype=float32)}}, tx=GradientTransformationExtraArgs(init=.init_fn at 0x7fe6daeca4d0>, update=.update_fn at 0x7fe6daeca9e0>), opt_state=(ScaleByAdamState(count=Array(0, dtype=int32), mu={'Dense_0': {'bias': Array([0., 0., 0., 0., 0.], dtype=float32), 'kernel': Array([[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.]], dtype=float32)}}, nu={'Dense_0': {'bias': Array([0., 0., 0., 0., 0.], dtype=float32), 'kernel': Array([[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.]], dtype=float32)}}), EmptyState()))
```

FrozenDicts:
```
state = TrainState.create(apply_fn=model.apply, params=flax.core.freeze(params), tx=optax.adam(1e-3))
state
```
```
TrainState(step=0, apply_fn=, params=FrozenDict({
Dense_0: {
kernel: Array([[ 0.37229332, -0.4265755 , -1.1151816 , -0.09558704, -0.62169886],
[-1.060781 , 1.0546707 , 0.33051118, -0.7090655 , 0.37682843],
[-0.30747807, -0.39064118, -0.25515485, 0.5127583 , -0.5559202 ]], dtype=float32),
bias: Array([0., 0., 0., 0., 0.], dtype=float32),
},
}), tx=GradientTransformationExtraArgs(init=.init_fn at 0x7fe710e21ea0>, update=.update_fn at 0x7fe710e225f0>), opt_state=(ScaleByAdamState(count=Array(0, dtype=int32), mu=FrozenDict({
Dense_0: {
bias: Array([0., 0., 0., 0., 0.], dtype=float32),
kernel: Array([[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.]], dtype=float32),
},
}), nu=FrozenDict({
Dense_0: {
bias: Array([0., 0., 0., 0., 0.], dtype=float32),
kernel: Array([[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.]], dtype=float32),
},
})), EmptyState()))
```

The indented representation can be viewed by calling [`flax.core.pretty_repr`](https://flax.readthedocs.io/en/latest/api_reference/flax.core.frozen_dict.html#flax.core.frozen_dict.pretty_repr) on the dict. Alternatively we could subclass dict and override the `__repr__` method to return an indented representation and have Flax return this subclass when `.init`, `.init_with_output` and `.apply` are called:
```
@flax.struct.dataclass
class MutableDict(dict):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def __repr__(self):
return 'MutableDict' + flax.core.pretty_repr(self)

state = TrainState.create(apply_fn=model.apply, params=MutableDict(params), tx=optax.adam(1e-3))
state
```
```
TrainState(step=0, apply_fn=, params=MutableDict{
Dense_0: {
kernel: Array([[ 0.37229332, -0.4265755 , -1.1151816 , -0.09558704, -0.62169886],
[-1.060781 , 1.0546707 , 0.33051118, -0.7090655 , 0.37682843],
[-0.30747807, -0.39064118, -0.25515485, 0.5127583 , -0.5559202 ]], dtype=float32),
bias: Array([0., 0., 0., 0., 0.], dtype=float32),
},
}, tx=GradientTransformationExtraArgs(init=.init_fn at 0x7fe710e3a5f0>, update=.update_fn at 0x7fe710e39990>), opt_state=(ScaleByAdamState(count=Array(0, dtype=int32), mu=MutableDict{}, nu=MutableDict{}), EmptyState()))
```
Another option is to add a section in the [dict migration guide](https://flax.readthedocs.io/en/latest/guides/regular_dict_upgrade_guide.html) to let users know they can get the indented representation by calling `flax.core.pretty_repr` (although this currently works only on FrozenDicts and regular dicts, and not other objects like TrainState).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.