ValueError: Precomputed metric requires shape (n_queries, n_indexed)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.3k
- Forks
- 871
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 5
Description
I just wanted to bring to your attention this error message. I believe this error is a little misleading because the algorithm works for n_neighbors=15 but not n_neighbors=3. Do you know what it could be in the backend that is preventing it from working for n_neighbors=3 and throwing the shape message?
umap.__version__
0.3.7
# Shape?
print(X.shape)
(5843, 5843)
# Symmetric?
def check_symmetric(a, tol=1e-8):
return np.allclose(a, a.T, atol=tol)
print(check_symmetric(X))
True
# Nulls?
print(np.any(X.isnull()))
False
# Diagonal?
print(np.unique(np.diagonal(X.values)))
[0.]
# UMAP Precomputed
model = UMAP(n_neighbors=3, metric="precomputed")
embeddings = model.fit_transform(X)
Error
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-40-44805956fe15> in <module>
18 # UMAP Precomputed
19 model = UMAP(n_neighbors=3, metric="precomputed")
---> 20 embeddings = model.fit_transform(X)
~/anaconda/envs/µ_env/lib/python3.6/site-packages/umap/umap_.py in fit_transform(self, X, y)
1564 Embedding of the training data in low-dimensional space.
1565 """
-> 1566 self.fit(X, y)
1567 return self.embedding_
1568
~/anaconda/envs/µ_env/lib/python3.6/site-packages/umap/umap_.py in fit(self, X, y)
1536 self.metric,
1537 self._metric_kwds,
-> 1538 self.verbose,
1539 )
1540
~/anaconda/envs/µ_env/lib/python3.6/site-packages/umap/umap_.py in simplicial_set_embedding(data, graph, n_components, initial_alpha, a, b, gamma, negative_sample_rate, n_epochs, init, random_state, metric, metric_kwds, verbose)
941 random_state,
942 metric=metric,
--> 943 metric_kwds=metric_kwds,
944 )
945 expansion = 10.0 / initialisation.max()
~/anaconda/envs/µ_env/lib/python3.6/site-packages/umap/spectral.py in spectral_layout(data, graph, dim, random_state, metric, metric_kwds)
238 random_state,
239 metric=metric,
--> 240 metric_kwds=metric_kwds,
241 )
242
~/anaconda/envs/µ_env/lib/python3.6/site-packages/umap/spectral.py in multi_component_layout(data, graph, n_components, component_labels, dim, random_state, metric, metric_kwds)
120 dim,
121 metric=metric,
--> 122 metric_kwds=metric_kwds,
123 )
124 else:
~/anaconda/envs/µ_env/lib/python3.6/site-packages/umap/spectral.py in component_layout(data, n_components, component_labels, dim, metric, metric_kwds)
51
52 distance_matrix = pairwise_distances(
---> 53 component_centroids, metric=metric, **metric_kwds
54 )
55 affinity_matrix = np.exp(-distance_matrix ** 2)
~/anaconda/envs/µ_env/lib/python3.6/site-packages/sklearn/metrics/pairwise.py in pairwise_distances(X, Y, metric, n_jobs, **kwds)
1381
1382 if metric == "precomputed":
-> 1383 X, _ = check_pairwise_arrays(X, Y, precomputed=True)
1384 return X
1385 elif metric in PAIRWISE_DISTANCE_FUNCTIONS:
~/anaconda/envs/µ_env/lib/python3.6/site-packages/sklearn/metrics/pairwise.py in check_pairwise_arrays(X, Y, precomputed, dtype)
118 "(n_queries, n_indexed). Got (%d, %d) "
119 "for %d indexed." %
--> 120 (X.shape[0], X.shape[1], Y.shape[0]))
121 elif X.shape[1] != Y.shape[1]:
122 raise ValueError("Incompatible dimension for X and Y matrices: "
ValueError: Precomputed metric requires shape (n_queries, n_indexed). Got (291, 5843) for 291 indexed.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at umap/umap_.py fit and spectral.py component_layout, following the traceback into sklearn's pairwise_distances handling for precomputed matrices. Reproduce the provided 5843x5843 case with n_neighbors=3 and 15; done means the cause of the shape error is established and the n_neighbors=3 behavior or its error message is made actionable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, scikit-learn
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100