ageron / ageron/handson-ml2

mask_zero in the Embedding layer

Abierto
#369 1 comentario 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Jupyter Notebook
Estrellas
30k
Forks
13.1k
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

Hello,

In the notebook corresponding to chapter 13, there is the following code at cell 156:

```python
model = keras.models.Sequential([
text_vectorization,
keras.layers.Embedding(input_dim=n_tokens,
output_dim=embedding_size,
mask_zero=True), # tokens => zero vectors
keras.layers.Lambda(compute_mean_embedding),
keras.layers.Dense(100, activation="relu"),
keras.layers.Dense(1, activation="sigmoid"),
])
```

The comment says that the indexes with value 0 (tokens) are mapped to zero vectors. However, I can't reproduce this behavior and after some googling in stackoverflow I found a confirmation that these zero indexes are not necessarily mapped to zero vectors.
https://stackoverflow.com/questions/47485216/how-does-mask-zero-in-keras-embedding-layer-work/53470422

If that is the case, how can we count the number of non-padding indexes(that is, the number of valid words in a sentence) after applying the Embedding layer?.

Thanks.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.