lmcinnes / lmcinnes/umap

Random Projection forest initialisation failed

Open
#583 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8.3k
Forks
871
Avg merge
1d 13h
Merged PRs (30d)
5

Description

Hi,

Since UMAP v5, I have been getting the following warning quite often when dealing with large dataset (>10k points, ~100 features) and it will end up getting stuck.. (I am running it on sever so i have almost unlimited computing resources)

`./myenv/lib/python3.7/site-packages/pynndescent/rp_trees.py:1005: UserWarning: Random Projection forest initialisation failed due to recursionlimit being reached. Something is a little strange with your graph_data, and this may take longer than normal to compute.
"Random Projection forest initialisation failed due to recursion"`

Any idea how to solve this issue? I would really appreciate your help. Thanks!

Contributor guide

Open the contributing guide

Research direction

Start at pynndescent/rp_trees.py around line 1005, where the recursion-limit warning is emitted. Reproduce the behavior with a dataset larger than 10,000 points and about 100 features using UMAP v5, then investigate why initialization reaches the recursion limit and hangs. Done means the reported warning and resulting prolonged computation are addressed for this case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.