mask_zero in the Embedding layer
- Dominant language
- Jupyter Notebook
- Stars
- 30k
- Forks
- 13.1k
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
In the notebook corresponding to chapter 13, there is the following code at cell 156:
```python
model = keras.models.Sequential([
text_vectorization,
keras.layers.Embedding(input_dim=n_tokens,
output_dim=embedding_size,
mask_zero=True), # tokens => zero vectors
keras.layers.Lambda(compute_mean_embedding),
keras.layers.Dense(100, activation="relu"),
keras.layers.Dense(1, activation="sigmoid"),
])
```
The comment says that the indexes with value 0 (tokens) are mapped to zero vectors. However, I can't reproduce this behavior and after some googling in stackoverflow I found a confirmation that these zero indexes are not necessarily mapped to zero vectors.
https://stackoverflow.com/questions/47485216/how-does-mask-zero-in-keras-embedding-layer-work/53470422
If that is the case, how can we count the number of non-padding indexes(that is, the number of valid words in a sentence) after applying the Embedding layer?.
Thanks.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.