Make redundant `features` argument optional for recurrent cells
- Dominant language
- Jupyter Notebook
- Stars
- 7.3k
- Forks
- 833
- Avg merge
- 5h 11m
- Merged PRs (30d)
- 5
Description
For recurrent cells such as the following:
- [`LSTMCell`](https://flax.readthedocs.io/en/latest/api_reference/flax.linen/_autosummary/flax.linen.LSTMCell.html)
- [`GRUCell`](https://flax.readthedocs.io/en/latest/api_reference/flax.linen/_autosummary/flax.linen.GRUCell.html)
- [`MGUCell`](https://flax.readthedocs.io/en/latest/api_reference/flax.linen/_autosummary/flax.linen.MGUCell.html)
- [`SimpleCell`](https://flax.readthedocs.io/en/latest/api_reference/flax.linen/_autosummary/flax.linen.SimpleCell.html)
- [`OptimizedLSTMCell`](https://flax.readthedocs.io/en/latest/api_reference/flax.linen/_autosummary/flax.linen.OptimizedLSTMCell.html)
the `features` argument of the constructor is redundant: It can be inferred from the `carry` input to its `__call__` method. (The only cell that currently uses `self.features` in its `__call__` method is [`ConvLSTMCell`](https://flax.readthedocs.io/en/latest/api_reference/flax.linen/_autosummary/flax.linen.ConvLSTMCell.html), which ought to be modified to infer it from its `carry` input.)
For each cell, the only place where `self.features` is needed is in the `initialize_carry` method. But in many models, the initial carry comes from "upstream" in the model, so this method is never used.
**Proposal:**
1. Edit `ConvLSTMCell` to infer `features` in its `__call__` method from its `carry` input.
2. Set `features=None` by default in each cell's constructor.
3. Add the following line to each `initialize_carry` method:
```python3
assert self.features is not None, "features cannot be None when calling initialize_carry"
```
I can submit a PR for this, if desired.
An alternative would be to pass `features` directly to the `initialize_carry` method.
Contributor guide
Assessment
This issue has not been assessed yet.