lmcinnes / lmcinnes/umap

How can we apply the Gower metric to UMAP?

Open
#356 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.3k
Forks
871
Avg merge
1d 13h
Merged PRs (30d)
5

Description

From my rough work, if we let the custom metric be the Gower metric, the distance matrix for all points in the dataset can be computed for both numerical and categorical data. However, it seems this is simplest when we only use the Gower metric for precomputing the distance matrix for the entire dataset, i.e. with

`umap.UMAP(metric="precomputed").fit_transform(precomputed_distances)`

While it is possible to compute the distance matrix for a dataset beforehand, using metric="precomputed" is inappropriate towards a further transform on the embedding for new data, which is needed for inference, since it doesn't allow for a .transform on the embedding for new data.

I think what I would want is to have a metric which can be plugged into umap.UMAP() such that this metric can handle both numerical and categorical features.

From the examples in the doc, it seems the metric is used for computing the distances between each pair of points separately (i.e. such a metric returns distance(point1, point2)),

**I'm wondering how one could use the Gower distance metric for both fitting against training data and transforming on test data?**

Or is transform for mixed datasets currently still unsupported despite the above?

This is important for me since I'm trying to use UMAP for dimensionality reduction on complex mixed datasets for inference/classification.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the umap.UMAP(metric="precomputed").fit_transform entry point and the documented custom-metric examples. Check how metric evaluation differs between fitting and transforming new data; done would be a documented supported path or a clearly stated limitation for mixed numerical and categorical data.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.