UMAP dataframe distance metrics for mixed type categorical data
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.3k
- Forks
- 871
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 5
Description
I recently came across [this paper](https://www.ncbi.nlm.nih.gov/pubmed/30908240) and it seems potentially very useful for the work going on in the `dataframe_umap` branch.
To summarize, the authors provide an entropy-based distance metric that can take any kind of categorical data (nominal *and* ordinal), with appropriate weighting that balances the relative information content for each attribute/column in your dataframe. I found the paper was written clearly: the key result is equation 15, and they provide a suggested algorithm structure too. Is this something you might consider for a PR?
**Paper:**
Zhang Y, Cheung YM, Tan KC. A Unified Entropy-Based Distance Metric for Ordinal-and-Nominal-Attribute Data Clustering. IEEE Trans Neural Netw Learn Syst. 2019 Mar 19. doi: 10.1109/TNNLS.2019.2899381.
https://www.ncbi.nlm.nih.gov/pubmed/30908240
Related previous discussions:
https://github.com/lmcinnes/umap/issues/58
https://github.com/lmcinnes/umap/issues/206
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the linked paper, especially equation 15 and its suggested algorithm, then review the dataframe_umap branch and the related discussions in issues 58 and 206. A complete result would be a scoped, reviewed PR adding the entropy-based metric for mixed nominal and ordinal dataframe data, with its expected behavior agreed beforehand.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100