dselivanov / dselivanov/text2vec

Skip Gram with Negative Sampling

Open
#193 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
R
Stars
876
Forks
133
PR merge metrics
No merged PRs in 30d

Description

https://arxiv.org/pdf/1705.09755v1.pdf

I recently posted a paper to arXiv showing that word2vec's Skip Gram with Negative Sampling (SGNS) algorithm is a weighted logistic PCA. With that framework, SGNS can be trained using the same term-context matrix that is used for GloVe. The training could use the same AdaGrad procedure, only with different gradients and loss function and sampling all of the elements of the matrix instead of just the non-zeroes.

Is SGNS something you are interested in including in the text2vec package, or are you happy with GloVe?

Thanks

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.