Reproducibility of results with transform method
- Dominant language
- Python
- Stars
- 8.3k
- Forks
- 871
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 5
Description
I have used UMAP for dimensionality reduction as a pre-processing step to build a clustering model. While trying to implement the model in a production environment, I have found that the results of the transform method are not reproducible, i.e. the same input data yield a significantly different result if fed into the model as part of different subsets of data. I would like to know if there is a way to force reproducibility of results with a saved UMAP model, or if there is a timeline to implement this in the library, because it performs very well for our use case compared to other methods, and so it would be a shame having to let it go because this issue makes it impossible to use in a production environment.
P.S. Many thanks for the library in any case, it's awesome!
Contributor guide
Research direction
Start by reproducing the saved UMAP model's transform results when identical inputs are presented in different subsets or batches. Inspect the transform method and establish a regression test that defines reproducible output across those inputs; the issue is done when the behavior is deterministic or the limitation is clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100