lmcinnes / lmcinnes/umap

Transform shifting clusters on repeated calls

Open
#247 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.3k
Forks
871
Avg merge
1d 13h
Merged PRs (30d)
5

Description

Hi @lmcinnes,
I am trying to use UMAP to show the increasing ability to seperate different classes over the training process of a neural network, similar to [https://medium.com/@rakesh.chada/understanding-neural-networks-by-embedding-hidden-representations-f256842ebf3a](url)

My idea was to select an epoch e, a layer l, as well as an input i.
I would then fit a UMAP instance on the high dimensional representations of i at layer l for that epoch e.
Afterwards, I want to use the learned transformation on other epochs for a fixed i and l.

However, the transformations for the same e, l and i seem to change after several runs on other e's. I don't mean some noisy variations, that I would expect to derive from the non-deterministic nature of the algorithm, but all the clusters are changing their shape and position within the embedding.

On the images you can see results of alternating transforms on epoch e=83 / 82 for fixed l and i.

![testing_repeated_umap_0](https://user-images.githubusercontent.com/20558123/58800903-6347fa80-8609-11e9-8db6-a85129b5ca80.png)
![testing_repeated_umap_1](https://user-images.githubusercontent.com/20558123/58800900-6347fa80-8609-11e9-8a1c-6bb8fd3ec201.png)
![testing_repeated_umap_2](https://user-images.githubusercontent.com/20558123/58800897-6347fa80-8609-11e9-8648-ca5a5e620199.png)
![testing_repeated_umap_3](https://user-images.githubusercontent.com/20558123/58800896-62af6400-8609-11e9-8858-b2a1f54d98a3.png)
![testing_repeated_umap_4](https://user-images.githubusercontent.com/20558123/58800893-62af6400-8609-11e9-94e4-30436587481e.png)
![testing_repeated_umap_5](https://user-images.githubusercontent.com/20558123/58800894-62af6400-8609-11e9-8e91-3128146b3a9e.png)
![testing_repeated_umap_6](https://user-images.githubusercontent.com/20558123/58800904-6347fa80-8609-11e9-9710-ca8fbad682a5.png)

Do you know, why the clusters are shifting so much and how I could possibly prevent this?
Thank you very much for your time.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the alternating fit/transform sequence described for epochs 82 and 83 with a fixed layer and input, then inspect the UMAP fit and transform entry points for state changes across repeated calls. Done means repeated transforms of the same fitted representation remain stable, or the stateful behavior is documented with a reliable explanation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-visualization, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.