lmcinnes / lmcinnes/umap

Does not work well on trained doc2vec model

Open
#5 7 comments 0 reactions 0 assignees View on GitHub
Good Reads
Dominant language
Python
Stars
8.3k
Forks
871
Avg merge
1d 13h
Merged PRs (30d)
5

Description

I trained a doc2vec model on the [large movie review dataset](http://ai.stanford.edu/~amaas/data/sentiment/) and then tried to use UMAP to reduce the dimensions of the resulting document vectors. I had hoped that it would be possible to separate the documents by sentiment (positive and negative), but unfortunately the embedding is one big blob. A notebook can be seen [here](https://github.com/gclen/umap-doc2vec-testing/blob/master/umap_with_imdb.ipynb) and the rest of the files for training the doc2vec model are in that [repository](https://github.com/gclen/umap-doc2vec-testing) as well.

Contributor guide

Open the contributing guide

Research direction

Start with the linked umap_with_imdb.ipynb and the linked repository's doc2vec training files; reproduce the reduction on the trained document vectors and compare the embedding with the reported sentiment expectation. Determine whether the blob reflects UMAP behavior or the input embeddings, then document the conclusion or next steps.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-visualization, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.