scikit-learn / scikit-learn/scikit-learn

Rationalize metric argument in T-SNE

Open
#9,695 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

module:manifold Needs Decision
Dominant language
Python
Stars
67.3k
Forks
27.4k
Avg merge
1d 15h
Merged PRs (30d)
58

Description

The API of T-SNE have a metric argument to allow changing the metric in the input space. The original paper was developed using only the euclidean distance and changing the metric might change a bit the math.

The main issue is to understand what is going on with the computation of the conditional probabilities. The original paper states that if d_ij is the euclidean distance between x_i and x_j, then the conditional probability is given by

p(x_j | x_i) = exp{ d_ij^2 / sig_i} / Z

where Z is a normalization constant and sig_i is a parameter, computed to fix the perplexity of the model. If d_ij is not the euclidean distance, does it make sense to use this formula?
The current implementation for other metric than the euclidean uses the formula without the square, so

p(x_j | x_i) = exp{ d_ij / sig_i} / Z

If it is fine, this should be documented. Else, shall we change it to use the square or another distribution? Another solution is to deprecate the metric argument.

A second point is the use of init='pca'. It is a very natural initialization for the euclidean metric but should be benchmarked for other to see if it gives an interesting starting point or not.

This issue is a follow-up of #9623 .

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no implementation files or tests. Start by locating the T-SNE implementation and its conditional-probability and PCA-initialization paths, then compare the current metric-dependent behavior with the cited formulation. Done means reaching and documenting a maintainer-backed decision on the metric API, probability calculation, and non-Euclidean PCA initialization.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
machine-learning
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.