shape of dist_matrix in MultiSelfAttention
Abierto
- Lenguaje dominante
- Python
- Estrellas
- 435
- Forks
- 118
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
In MultiSelfAttention class:
line 235: dist_matrix = tf.einsum("bwhd,bkhd->bwkh", queries, keys) # dots of (batch, word, key, head)
line 255: select_probs = tf.nn.softmax(dist_matrix) # for each (batch, word, head) probability over keys
The shapes of dist_matrix in the two lines seem not consistent. Should line 235 be changed to:
dist_matrix = tf.einsum("bwhd,bkhd->bwhk", queries, keys) # dots of (batch, word, head, key) ?
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.