Reproducibility Discrepancy in UMAP in Local Environment vs Deployed App
- Dominant language
- Python
- Stars
- 8.3k
- Forks
- 871
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 5
Description
Hello,
I am writing to report a reproducibility issue encountered while dockerizing my app, which utilizes UMAP. Despite ensuring the use of the exact same dependencies' version and UMAP's hyperparameters, I observed different results between my Jupyter notebook and the deployed app on an AWS instance.
To illustrate the disparity, I have provided a screenshot:

`umap_embeddings` represents the result obtained in the notebook, while `umap_embeddings_job` is the outcome when running the container on the AWS instance.
The code used in both environments is as follows:
```
docs = df['text'].str.lower().str.replace("\n", " ").values
embedding_model_sbert = SentenceTransformer(SENTENCE_TRANSFORMER_MODEL)
embeddings_sbert = embedding_model_sbert.encode(docs, show_progress_bar=True)
umap_model = UMAP(**UMAP_PARAMS)
umap_embeddings = umap_model.fit_transform(embeddings_sbert)
```
where the parameters used in UMAP are defined in this dictionary:
```
UMAP_PARAMS = dict(
n_neighbors=15,
n_components=5,
min_dist=0.0,
metric='cosine',
random_state=42
)
```
In both environments I am using Python version 3.8.10 and UMAP version 0.5.3. The list of UMAP's dependencies installed in both environments is as follows:
```
numpy==1.23.0
scipy==1.8.0
pynndescent==0.5.10
scikit-learn==1.0.2
```
Am I missing something?
Thank you in advance!
Contributor guide
Research direction
No repository file or test is named. Start by comparing the notebook and AWS container at the UMAP fit_transform call, including the listed Python, UMAP, and dependency versions and the generated embeddings; done means identifying why the environments diverge and documenting or validating reproducible matching results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, docker, jupyter-notebook, numpy, python, scikit-learn
- Domain
- cloud, devops, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100