ageron / ageron/handson-ml2

mask_zero in the Embedding layer

Open
#369 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
30k
Forks
13.1k
PR merge metrics
No merged PRs in 30d

Description

Hello,

In the notebook corresponding to chapter 13, there is the following code at cell 156:

```python
model = keras.models.Sequential([
text_vectorization,
keras.layers.Embedding(input_dim=n_tokens,
output_dim=embedding_size,
mask_zero=True), # tokens => zero vectors
keras.layers.Lambda(compute_mean_embedding),
keras.layers.Dense(100, activation="relu"),
keras.layers.Dense(1, activation="sigmoid"),
])
```

The comment says that the indexes with value 0 (tokens) are mapped to zero vectors. However, I can't reproduce this behavior and after some googling in stackoverflow I found a confirmation that these zero indexes are not necessarily mapped to zero vectors.
https://stackoverflow.com/questions/47485216/how-does-mask-zero-in-keras-embedding-layer-work/53470422

If that is the case, how can we count the number of non-padding indexes(that is, the number of valid words in a sentence) after applying the Embedding layer?.

Thanks.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.