lmcinnes / lmcinnes/umap

segmentation fault issue depending on hyperparameter

Open
#399 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.3k
Forks
871
Avg merge
1d 13h
Merged PRs (30d)
5

Description

Hello.

I was facing weird occasions where UMAP works perfectly for a set of hyperparameter while another set of hyperparameter causing error exhibiting 'segmentation fault'. It is not due to data itself nor virtualenv since they were identical. FYI, data is about (500,000 x 39) numpy array.

The versions of packages are:
- umap-learn == 0.4.0 (but was also same in 0.3.0 and 0.4.0rc3).
- numba ==0.48.0 (I tried 0.40.0, 0.44.0, 0.46.0 but the result was identical)
- scikit-learn ==0.22.0
- numpy==1.18.2
- scipy == 1.4.0
- llvmlite == 0.31.0
- joblib==0.14.1
- python=3.6.1 (I tested with python=3.6.4 also)

The functioning hyperparameter set is:
- n_neighbor: 30, 50 (I did not search all yet)
- min_dist: 0.005 to 0.5
- metric: correlation
- dimension:2

The malfunctioning hyperparameter set it
-n_neighbor: 70, 90, 110 (I did not search all yet)
-min_dist: 0.005 to 0.5 (these are identical from above)
-metric: correlation
-dimension:2

Just to add, the OS system is Linux Mint 18.4. I tested both on the command line by .py and Jupyter lab.

If there are any guess why it might happen, please let me know. Thank you.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue from the command-line .py example using the reported 500,000 × 39 NumPy array and compare the working and failing n_neighbor values. Check whether the same behavior occurs in Jupyter Lab, then narrow down the parameter combination and environment involved. Done means the segmentation fault is explained and a verified fix or reproducible diagnosis is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python, scikit-learn
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.