Productionalizing UMAP Embedded points for new data
- Dominant language
- Python
- Stars
- 8.3k
- Forks
- 871
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 5
Description
Hello! I am using umap to reduce a mixed dataset (numerical and categorical features) for a clustering analysis.
I currently have the following code which has two fits, one for numerical, and one for categorical features.
```
#categorical
categorical = pd.get_dummies(categorical)
fit2_full = umap.UMAP(metric = 'dice'
,n_neighbors = 30
,min_dist = 0.0).fit(categorical)
#numerical
fit1_full = umap.UMAP(metric = 'l2',
n_neighbors = 30
,min_dist = 0.0).fit(numerical)
# intersection
intersection = umap.general_simplicial_set_intersection(fit1_full.graph_, fit2_full.graph_, weight=categorical_weight)
intersection = umap.reset_local_connectivity(intersection)
embedding = umap.simplicial_set_embedding(fit1_full._raw_data, intersection, fit1_full.n_components,
fit1_full._initial_alpha, fit1_full._a, fit1_full._b,
fit1_full.repulsion_strength, fit1_full.negative_sample_rate,
200, 'random', np.random, fit1_full.metric,
fit1_full._metric_kwds, False
,output_dens = False
,densmap_kwds = {})
```
If I want to productionalize this, and apply the fitted umap to new data, how would I do this? Sorry for the formatting of this post, I'm a novice at raising issues. Any help is appreciated! Thank you
Contributor guide
Research direction
Start from the two UMAP fits and the subsequent general_simplicial_set_intersection and simplicial_set_embedding calls shown in the issue. Determine how the fitted mixed-feature embedding should be applied to new data, and consider the work complete when new numerical and categorical rows can be embedded consistently in production.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, pandas, python
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100