Umap map samples from same data distribution into different region
- Dominant language
- Python
- Stars
- 8.3k
- Forks
- 871
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 5
Description
It seems like the umap model did not map the same data distribution samples to the same general region. This problem does not seems to be present on rapids umap implementation (which is weird)
Here are the sample code
**Data Generation**
```python
import numpy as np
n_samples = 10000
n_dims = 5
data = []
np.random.seed(1337)
for _ in range(n_dims):
data.append(np.random.normal(loc=(5*np.random.uniform() + 3), scale=2.0, size=n_samples))
order = np.arange(data[-1].shape[0])
np.random.shuffle(order)
order = order[:int(0.5*np.random.uniform()*order.shape[0])]
data[-1][order] = 0.0
data[-1][data[-1] < 0] = 0.0
data = np.array(data).T
train_data = data[:int(len(data)*0.5)]
test_data = data[int(len(data)*0.5):]
```
**Umap**
```python
import umap
fitter = umap.UMAP(
n_neighbors=30,
min_dist=0.0,
n_components=2,
negative_sample_rate=15,
random_state=42,
verbose=True,
).fit(train_data)
train_embedding = fitter.transform(train_data)
test_embedding = fitter.transform(test_data)
```
**Plot**
```python
import matplotlib.pyplot as plt
plt.style.use('ggplot')
fig, ax = plt.subplots(1, figsize=(15, 15))
plt.scatter(*train_embedding.T, s=3, alpha=0.5, label="train")
plt.scatter(*test_embedding.T, s=3, alpha=0.5, label="test")
plt.legend(loc="best")
plt.setp(ax, xticks=[], yticks=[])
plt.show()
```
Umap (CPU)

Umap (Rapids - GPU)

Contributor guide
Research direction
Start by running the supplied Python reproduction using umap.UMAP.fit on train_data and transform for both train_data and test_data, then compare the plotted embeddings. Investigate whether the CPU behavior differs from the reported RAPIDS result; done should establish the cause and define whether a fix or documented limitation is needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100