lmcinnes / lmcinnes/umap

UMAP dataframe distance metrics for mixed type categorical data

Open
#303 1 comment 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.3k
Forks
871
Avg merge
1d 13h
Merged PRs (30d)
5

Description

I recently came across [this paper](https://www.ncbi.nlm.nih.gov/pubmed/30908240) and it seems potentially very useful for the work going on in the `dataframe_umap` branch.

To summarize, the authors provide an entropy-based distance metric that can take any kind of categorical data (nominal *and* ordinal), with appropriate weighting that balances the relative information content for each attribute/column in your dataframe. I found the paper was written clearly: the key result is equation 15, and they provide a suggested algorithm structure too. Is this something you might consider for a PR?

**Paper:**
Zhang Y, Cheung YM, Tan KC. A Unified Entropy-Based Distance Metric for Ordinal-and-Nominal-Attribute Data Clustering. IEEE Trans Neural Netw Learn Syst. 2019 Mar 19. doi: 10.1109/TNNLS.2019.2899381.
https://www.ncbi.nlm.nih.gov/pubmed/30908240

Related previous discussions:
https://github.com/lmcinnes/umap/issues/58
https://github.com/lmcinnes/umap/issues/206

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the linked paper, especially equation 15 and its suggested algorithm, then review the dataframe_umap branch and the related discussions in issues 58 and 206. A complete result would be a scoped, reviewed PR adding the entropy-based metric for mixed nominal and ordinal dataframe data, with its expected behavior agreed beforehand.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.