lmcinnes / lmcinnes/umap

Productionalizing UMAP Embedded points for new data

Open
#982 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8.3k
Forks
871
Avg merge
1d 13h
Merged PRs (30d)
5

Description

Hello! I am using umap to reduce a mixed dataset (numerical and categorical features) for a clustering analysis.

I currently have the following code which has two fits, one for numerical, and one for categorical features.

```
#categorical
categorical = pd.get_dummies(categorical)
fit2_full = umap.UMAP(metric = 'dice'
,n_neighbors = 30
,min_dist = 0.0).fit(categorical)

#numerical
fit1_full = umap.UMAP(metric = 'l2',
n_neighbors = 30
,min_dist = 0.0).fit(numerical)

# intersection
intersection = umap.general_simplicial_set_intersection(fit1_full.graph_, fit2_full.graph_, weight=categorical_weight)
intersection = umap.reset_local_connectivity(intersection)
embedding = umap.simplicial_set_embedding(fit1_full._raw_data, intersection, fit1_full.n_components,
fit1_full._initial_alpha, fit1_full._a, fit1_full._b,
fit1_full.repulsion_strength, fit1_full.negative_sample_rate,
200, 'random', np.random, fit1_full.metric,
fit1_full._metric_kwds, False
,output_dens = False
,densmap_kwds = {})

```

If I want to productionalize this, and apply the fitted umap to new data, how would I do this? Sorry for the formatting of this post, I'm a novice at raising issues. Any help is appreciated! Thank you

Contributor guide

Open the contributing guide

Research direction

Start from the two UMAP fits and the subsequent general_simplicial_set_intersection and simplicial_set_embedding calls shown in the issue. Determine how the fitted mixed-feature embedding should be applied to new data, and consider the work complete when new numerical and categorical rows can be embedded consistently in production.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
data, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.