lmcinnes / lmcinnes/umap

Reproducibility of results with transform method

Open
#950 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8.3k
Forks
871
Avg merge
1d 13h
Merged PRs (30d)
5

Description

I have used UMAP for dimensionality reduction as a pre-processing step to build a clustering model. While trying to implement the model in a production environment, I have found that the results of the transform method are not reproducible, i.e. the same input data yield a significantly different result if fed into the model as part of different subsets of data. I would like to know if there is a way to force reproducibility of results with a saved UMAP model, or if there is a timeline to implement this in the library, because it performs very well for our use case compared to other methods, and so it would be a shame having to let it go because this issue makes it impossible to use in a production environment.

P.S. Many thanks for the library in any case, it's awesome!

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the saved UMAP model's transform results when identical inputs are presented in different subsets or batches. Inspect the transform method and establish a regression test that defines reproducible output across those inputs; the issue is done when the behavior is deterministic or the limitation is clearly documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.