google / google/flax

Clarification for LSTMCell Documentation

Open
#4,124 2 comments 0 reactions 0 assignees View on GitHub
Priority: P2 - no schedule
Dominant language
Jupyter Notebook
Stars
7.3k
Forks
833
Avg merge
5h 11m
Merged PRs (30d)
5

Description

Hello, I was trying to understand the [LSTMCell](https://flax.readthedocs.io/en/v0.8.0/api_reference/flax.linen/_autosummary/flax.linen.LSTMCell.html) of Flax. The documentation for the ```__call__``` function says:

```
carry – the hidden state of the LSTM cell, initialized using LSTMCell.initialize_carry.
```

It thought it was weird that the cell state wasn't returned in addition with the hidden state. But in the [source code](https://flax.readthedocs.io/en/v0.8.0/_modules/flax/linen/recurrent.html#LSTMCell), ```initialize_carry``` seems to return a tuple containing the cell and the hidden states :

```python
return (c, h)
```

Additionally, the ```__call__``` function seems to also returns both the cell state and the hidden state in the carry:

```python
return (new_c, new_h), new_h
```

Did I misunderstand something? If not, should the documentation be updated to clarify that the carry includes both the cell state and the hidden state?

Anyway, thanks for the great library!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.