Trainable Embedding for OOV words

Hello,

I have one doubt over your code: in your code, all OOV words are represented by id 1, which means, all OOV words are considered the same word, and its embedding is a zero vector. Also, this embedding will not be updated during training. However, in the original paper, the author mentioned that for OOV words, the word embeddings are updated during training.

I think this may be a reason why the score is lower than the original paper.

Trace how out-of-vocabulary words are assigned IDs and how their embeddings are initialized and updated during training. Compare that behavior with the original paper, then verify that OOV embeddings can train and that the resulting evaluation behavior is checked against the reported discrepancy.

Written by the indexing model from the issue text.