dselivanov / dselivanov/text2vec
Skip Gram with Negative Sampling
Open
- Dominant language
- R
- Stars
- 876
- Forks
- 133
- PR merge metrics
- No merged PRs in 30d
Description
https://arxiv.org/pdf/1705.09755v1.pdf
I recently posted a paper to arXiv showing that word2vec's Skip Gram with Negative Sampling (SGNS) algorithm is a weighted logistic PCA. With that framework, SGNS can be trained using the same term-context matrix that is used for GloVe. The training could use the same AdaGrad procedure, only with different gradients and loss function and sampling all of the elements of the matrix instead of just the non-zeroes.
Is SGNS something you are interested in including in the text2vec package, or are you happy with GloVe?
Thanks
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.