lmcinnes / lmcinnes/umap

ValueError: Precomputed metric requires shape (n_queries, n_indexed)

Open
#190 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.3k
Forks
871
Avg merge
1d 13h
Merged PRs (30d)
5

Description

I just wanted to bring to your attention this error message. I believe this error is a little misleading because the algorithm works for n_neighbors=15 but not n_neighbors=3. Do you know what it could be in the backend that is preventing it from working for n_neighbors=3 and throwing the shape message?

umap.__version__
0.3.7

# Shape?
print(X.shape)
​(5843, 5843)

# Symmetric?
def check_symmetric(a, tol=1e-8):
    return np.allclose(a, a.T, atol=tol)
print(check_symmetric(X))
​True

# Nulls?
print(np.any(X.isnull()))
​False

# Diagonal? 
print(np.unique(np.diagonal(X.values)))
​[0.]

# UMAP Precomputed
model = UMAP(n_neighbors=3, metric="precomputed")
embeddings = model.fit_transform(X)

Error

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-40-44805956fe15> in <module>
     18 # UMAP Precomputed
     19 model = UMAP(n_neighbors=3, metric="precomputed")
---> 20 embeddings = model.fit_transform(X)

~/anaconda/envs/µ_env/lib/python3.6/site-packages/umap/umap_.py in fit_transform(self, X, y)
   1564             Embedding of the training data in low-dimensional space.
   1565         """
-> 1566         self.fit(X, y)
   1567         return self.embedding_
   1568 

~/anaconda/envs/µ_env/lib/python3.6/site-packages/umap/umap_.py in fit(self, X, y)
   1536             self.metric,
   1537             self._metric_kwds,
-> 1538             self.verbose,
   1539         )
   1540 

~/anaconda/envs/µ_env/lib/python3.6/site-packages/umap/umap_.py in simplicial_set_embedding(data, graph, n_components, initial_alpha, a, b, gamma, negative_sample_rate, n_epochs, init, random_state, metric, metric_kwds, verbose)
    941             random_state,
    942             metric=metric,
--> 943             metric_kwds=metric_kwds,
    944         )
    945         expansion = 10.0 / initialisation.max()

~/anaconda/envs/µ_env/lib/python3.6/site-packages/umap/spectral.py in spectral_layout(data, graph, dim, random_state, metric, metric_kwds)
    238             random_state,
    239             metric=metric,
--> 240             metric_kwds=metric_kwds,
    241         )
    242 

~/anaconda/envs/µ_env/lib/python3.6/site-packages/umap/spectral.py in multi_component_layout(data, graph, n_components, component_labels, dim, random_state, metric, metric_kwds)
    120             dim,
    121             metric=metric,
--> 122             metric_kwds=metric_kwds,
    123         )
    124     else:

~/anaconda/envs/µ_env/lib/python3.6/site-packages/umap/spectral.py in component_layout(data, n_components, component_labels, dim, metric, metric_kwds)
     51 
     52     distance_matrix = pairwise_distances(
---> 53         component_centroids, metric=metric, **metric_kwds
     54     )
     55     affinity_matrix = np.exp(-distance_matrix ** 2)

~/anaconda/envs/µ_env/lib/python3.6/site-packages/sklearn/metrics/pairwise.py in pairwise_distances(X, Y, metric, n_jobs, **kwds)
   1381 
   1382     if metric == "precomputed":
-> 1383         X, _ = check_pairwise_arrays(X, Y, precomputed=True)
   1384         return X
   1385     elif metric in PAIRWISE_DISTANCE_FUNCTIONS:

~/anaconda/envs/µ_env/lib/python3.6/site-packages/sklearn/metrics/pairwise.py in check_pairwise_arrays(X, Y, precomputed, dtype)
    118                              "(n_queries, n_indexed). Got (%d, %d) "
    119                              "for %d indexed." %
--> 120                              (X.shape[0], X.shape[1], Y.shape[0]))
    121     elif X.shape[1] != Y.shape[1]:
    122         raise ValueError("Incompatible dimension for X and Y matrices: "

ValueError: Precomputed metric requires shape (n_queries, n_indexed). Got (291, 5843) for 291 indexed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at umap/umap_.py fit and spectral.py component_layout, following the traceback into sklearn's pairwise_distances handling for precomputed matrices. Reproduce the provided 5843x5843 case with n_neighbors=3 and 15; done means the cause of the shape error is established and the n_neighbors=3 behavior or its error message is made actionable.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.