lmcinnes / lmcinnes/umap

SpectralEmbedding warns "Graph is not fully connected"

Open
#847 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.3k
Forks
871
Avg merge
1d 13h
Merged PRs (30d)
5

Description

When learning an embedding with `UMAP(min_dist=0.0, n_neighbors=100).fit(data)`, `SpectralEmbedding` warns

```UserWarning: Graph is not fully connected, spectral embedding may not work as expected.```

It appears that the warning is issued regardless of selection of n_neighbors (only a small selection were attempted). Is the warning of concern? As this only happens with certain data, is there characteristics I can test the data for? If this occurs is `init = 'random'` the recourse?

Much appreciated.

~ Tom

Packages:

umap-learn 0.5.2
numpy 1.21.5
numba 0.55.1
scipy 1.8.0
scikit-learn 1.0.2
pynndescent 0.5.6

A MWE and data are attached:

```python
import numpy as np
import pandas as pd
import umap

print('-'*10, 'Loading', '-'*10)
dataframe = pd.read_csv('input.csv.gz', index_col=None, header=None)
data = dataframe.to_numpy()
### project using umap
print('-'*10, 'Projecting', '-'*10)
projection = umap.UMAP(n_neighbors=100, min_dist=0., n_components=2, init='spectral', verbose=True).fit_transform(data)
print('-'*10, 'Saving', '-'*10)
pd.DataFrame(projection).to_csv('output.csv.gz', header=False, index=False)
```
[input.csv.gz](https://github.com/lmcinnes/umap/files/8318581/input.csv.gz)
[affinity_matrix.npy.gz](https://github.com/lmcinnes/umap/files/8318588/affinity_matrix.npy.gz)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the attached MWE using input.csv.gz and inspect the reported SpectralEmbedding warning during UMAP(..., init='spectral'). Compare the attached affinity_matrix.npy.gz and the attempted n_neighbors values. Done would require a reproducible diagnosis and a documented or implemented resolution, but the issue does not specify which change is wanted.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python, scikit-learn
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.