MaartenGr / MaartenGr/BERTopic

Suggestion: Clip default hierarchy distance function cosine similarity

Open
#1,573 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

It appears that the sklearn `cosine_similarity` function can sometimes output values just slightly above 1. This has been reported and is under discussion within scikit-learn (https://github.com/scikit-learn/scikit-learn/issues/18122). This has led to a few submitted issues reported on this repo (https://github.com/MaartenGr/BERTopic/issues/1137, https://github.com/MaartenGr/BERTopic/issues/1418, and https://github.com/MaartenGr/BERTopic/issues/1319 for example) with a suggested workaround of using the absolute value of the default distance function.

Based on the direction it looks like they may take in the scikit-learn issue to address this I am proposing a slight change to the default distance function to clip the output of the cosine similarity function to the expected value ranges of -1 to 1 which would eliminate this issue and possibly prevent more issues from being reported.

```python
if distance_function is None:
distance_function = lambda x: 1 - cosine_similarity(x)
```

would become

```python
if distance_function is None:
distance_function = lambda x: 1 - np.clip(cosine_similarity(x), -1, 1)
```

I know that if https://github.com/scikit-learn/scikit-learn/issues/18122 is fixed this will no longer be necessary and we can always supply the custom distance function ourselves. Therefore I totally understand if adding this workaround isn't considered necessary right now but I wanted to propose this regardless to at least hear opinions on this approach. If this is something that is desired I'm happy to create the PR.

My understanding of cosine similarity is that there is no reason for the values to be outside of the -1 to 1 range besides floating point rounding errors so there is no risk to the behavior in clipping the output. Is this correct?

Thank you!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the code that assigns the default distance function and review its use of scikit-learn's cosine_similarity. Verify the proposed clipping range of -1 to 1 and check the surrounding behavior for custom distance functions. Done means the default path prevents out-of-range cosine similarity values without changing custom-distance behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.