lmcinnes / lmcinnes/umap

Multi CPU / GPU capabilities?

Open
#37 35 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8.3k
Forks
871
Avg merge
1d 13h
Merged PRs (30d)
5

Description

@lmcinnes
As you may have guessed I have several CPUs and GPUs at hand and I work with high-dimensional data.
Now I am benching a 500k * 5k => 500k * 2 vector vs. PCA (I need a high level clustering to filter my data to feed it further in the pipeline).

So a couple of questions:
1. Any plans on multi-CPU / GPU support?
2. Does your implementation utilize vectorized operations (not really sure how embedding methods like T-SNE and UMAP work, I believe they minimize some kind of distance in high dimension space?) If so, can I help?
3. Did you run benchmarks (like HDBSCAN) on large and huge datasets? If so, then is it feasible to expect 500k * 5k => 500k * 2 to finish in reasonable time, or should I do PCA => UMAP?

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. Start by locating the UMAP implementation and any existing benchmark coverage, then investigate the requested multi-CPU/GPU support, vectorization, and 500k × 5k workload; done would require a clearly scoped change or maintainer-backed benchmark conclusion.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.