Random Projection forest initialisation failed
- Dominant language
- Python
- Stars
- 8.3k
- Forks
- 871
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 5
Description
Hi,
Since UMAP v5, I have been getting the following warning quite often when dealing with large dataset (>10k points, ~100 features) and it will end up getting stuck.. (I am running it on sever so i have almost unlimited computing resources)
`./myenv/lib/python3.7/site-packages/pynndescent/rp_trees.py:1005: UserWarning: Random Projection forest initialisation failed due to recursionlimit being reached. Something is a little strange with your graph_data, and this may take longer than normal to compute.
"Random Projection forest initialisation failed due to recursion"`
Any idea how to solve this issue? I would really appreciate your help. Thanks!
Contributor guide
Research direction
Start at pynndescent/rp_trees.py around line 1005, where the recursion-limit warning is emitted. Reproduce the behavior with a dataset larger than 10,000 points and about 100 features using UMAP v5, then investigate why initialization reaches the recursion limit and hangs. Done means the reported warning and resulting prolonged computation are addressed for this case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100