lmcinnes / lmcinnes/umap

Parametric UMAP run time performance on GPU

Open
#869 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.3k
Forks
871
Avg merge
1d 13h
Merged PRs (30d)
5

Description

Currently, I am testing the Parametric UMAP on the GPU and I was expecting the performance lift compare to CPU performance.

But it does seem like the run time is not improved at all or even worse.

Here is my setup:
Instance Type - g5.16xlarge
tf version: 2.8
training data set size: 817,614 with 107 columns (after one hot it become 362 columns) - with intention to run 10 times large size later. full data set size is 80MM +
`Parametric parameters:
keras_fit_kwargs = {"callbacks": [
tf.keras.callbacks.EarlyStopping(
monitor='loss',
min_delta=10**-2,
patience=10,
verbose=1,
)
]}

embedder = ParametricUMAP(verbose=True, batch_size = 512, ##512
keras_fit_kwargs = keras_fit_kwargs,
n_training_epochs = 10)

with tf.device(gpus[0].name): ## '/device:GPU:0'
umap_features = embedder.fit_transform(df_train_transformed)`

both GPU and CPU never get to close on the finish line but based on estimated epoche time: CPU estimated 7 mins versus GPU estimated 13 mins.

Any tutorial/best practice to speed up running Parametric UMAP would be highly appreciated! (either GPU or CPU) thanks a lot!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The report names ParametricUMAP.fit_transform as the entry point but no repository file or test. First reproduce the CPU/GPU timing with the supplied dataset and TensorFlow setup, then trace that entry point to identify the runtime bottleneck. Done means a specific, reproducible cause and an agreed performance change or documented best practice.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.