seq2seq: Replace the embeddings with pre-trained word embeddings such as word2vec
Open
Nobody has claimed this yet.
medium
Text
- Dominant language
- Python
- Stars
- 9.3k
- Forks
- 4.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 4
Description
Hi,
Thank you for your tutorial! I tried to change the embedding with pre-trained word embeddings such as word2vec, here is my code:
class Lang:
def __init__(self, name):
self.name = name
self.word2index = {}
self.word2count = {}
self.index2word = {0: "SOS", 1: "EOS"}
self.n_words = 2 # Count SOS and EOS
def get_word2vec(self):
word2vec = KeyedVectors.load_word2vec_format('Models/Word2Vec/wiki.he.vec')
return word2vec
def addSentence(self, sentence):
for word in sentence.split(' '):
self.addWord(word)
def addWord(self, word):
if word not in self.word2index:
self.word2index[word] = self.get_word2vec[word]
self.word2count[word] = 1
self.index2word[self.n_words] = word
self.n_words += 1
else:
self.word2count[word] += 1
the dimension size of this word2vec is 300 dimensions
Is I need to change other things in my Encoder?
Thank you!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Lang class and the Encoder referenced in the issue, and review how the current embedding dimensions and word indexes are used. The issue does not name a file or test, and completion would require defining which encoder and data-pipeline changes are needed for pre-trained word embeddings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100