Random_state produces different results on different operating systems
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.3k
- Forks
- 871
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 5
Description
### Issue
The `random_state` parameter produces deterministic results on a specific OS, but does not produce the same results on different OSes. Here are some examples for umap-learn, run with the following code. I used the example from the README here, as well as Scikit's check_random_state as a control (all Scikit results are the same).
The results are also seem to be dependent on the version of Numba that is installed.
```
# UMAP example with random state
import umap
from sklearn.datasets import load_digits
digits = load_digits()
embedding = umap.UMAP(
n_neighbors=5,
min_dist=0.3,
metric='correlation',
random_state=2018,
).fit_transform(digits.data)
embedding
# Scikit check random state
from sklearn.utils import check_random_state
random_state = check_random_state(2018)
random_state.rand(4)
```
### Example Results
Machine | Architecture | Python Version | umap-learn Version | numba Version | UMAP Result
-- | -- | -- | -- | -- | --
Macbook Pro #1 | Darwin C02P141DG3QD 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 21 20:07:39 PDT 2018; root:xnu-3789.73.14~1/RELEASE_X86_64 x86_64 | Python 3.7.0 | 0.3.2 | 0.39.0 | array([[16.42446 , -2.1266642], [ 7.231049 , -1.5276358], [-1.5864906, -5.1226635], ..., [ 6.094945 , 1.2291753], [ 1.3193432, 5.4169164], [ 5.5729628, 2.2857437]], dtype=float32)
Macbook Pro #1 | Darwin C02P141DG3QD 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 21 20:07:39 PDT 2018; root:xnu-3789.73.14~1/RELEASE_X86_64 x86_64 | Python 3.7.0 | 0.3.5 | 0.40.1 | array([[32.471622, 8.842674], [16.400652, 13.036578], [ 9.181449, 3.948576], ..., [19.216055, 12.42009 ], [ 6.522507, 14.285691], [19.517092, 11.733169]], dtype=float32)
Macbook Pro #2 | Darwin C02VN4T7HV2L 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64 | Python 3.7.0 | 0.3.2 | 0.40.0 | array([[16.42446 , -2.1266642], [ 7.231049 , -1.5276358], [-1.5864906, -5.1226635], ..., [ 6.094945 , 1.2291753], [ 1.3193432, 5.4169164], [ 5.5729628, 2.2857437]], dtype=float32)
Macbook Pro #2 | Darwin C02VN4T7HV2L 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64 | Python 3.7.0 | 0.3.5 | 0.40.1 | array([[32.471622, 8.842674], [16.400652, 13.036578], [ 9.181449, 3.948576], ..., [19.216055, 12.42009 ], [ 6.522507, 14.285691], [19.517092, 11.733169]], dtype=float32)
Debian Docker | Linux 389088ec7b25 4.9.93-linuxkit-aufs #1 SMP Wed Jun 6 16:55:56 UTC 2018 x86_64 GNU/Linux | Python 3.5.3 | 0.3.5 | 0.40.1 | array([[25.864304 , 7.870304 ], [16.924606 , 7.9489594], [ 7.4818945, 9.081071 ], ..., [15.565144 , 10.721824 ], [ 7.7764506, 14.354664 ], [14.85415 , 11.515898 ]], dtype=float32)
Ubuntu Docker | Linux 6a9a07ef70b7 4.9.93-linuxkit-aufs #1 SMP Wed Jun 6 16:55:56 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | Python 3.6.6 | 0.3.5 | 0.40.1 | array([[25.864304 , 7.870304 ], [16.924606 , 7.9489594], [ 7.4818945, 9.081071 ], ..., [15.565144 , 10.721824 ], [ 7.7764506, 14.354664 ], [14.85415 , 11.515898 ]], dtype=float32)
Ubuntu Docker | Linux 6a9a07ef70b7 4.9.93-linuxkit-aufs #1 SMP Wed Jun 6 16:55:56 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | Python 3.7.0 | 0.3.5 | 0.40.1 | array([[25.864304 , 7.870304 ], [16.924606 , 7.9489594], [ 7.4818945, 9.081071 ], ..., [15.565144 , 10.721824 ], [ 7.7764506, 14.354664 ], [14.85415 , 11.515898 ]], dtype=float32)
Ubuntu Desktop | Linux brick 4.15.0-36-generic #39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | Python 3.6.6 | 0.3.5 | 0.40.1 | array([[25.864225 , 7.8703256], [16.92632 , 7.943247 ], [ 7.4819674, 9.081023 ], ..., [15.570685 , 10.72381 ], [ 7.776701 , 14.354493 ], [14.864248 , 11.530873 ]], dtype=float32)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the README UMAP example with random_state=2018 and compare it with sklearn's check_random_state control across the listed operating systems and versions. Investigate the UMAP and Numba version differences shown in the examples; done means the source of cross-OS and version-dependent results is identified and reproducibility expectations are established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100