Speed up Levenshtein distance calculation when we don't need the exact distance [LUCENE-9289]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Sometimes when we calculate the Levenshtein distance we don't need the exact distance, we only want to know if the strings are similar enough.
```
sug.score = sd.getDistance(original, sug.string);
if (sug.score < min) continue;
```
If we use this threshold in the distance calculation, we can speed it up, we can stop the calculation when we already know that the the the distance will be lower than the threshold.
---
Migrated from [LUCENE-9289](https://issues.apache.org/jira/browse/LUCENE-9289) by Andras Salamon (@asalamon74), updated Mar 30 2020
Attachments: [SOLR-14360-01.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-9289/SOLR-14360-01.patch)
Contributor guide
Research direction
Start at solr/core/src/java/org/apache/solr/spelling/SolrSpellChecker.java around lines 113-114, where getDistance is called and compared with min. Trace the distance implementation and determine how that threshold can be used during calculation. Done means the similarity check can stop when the result is already below the threshold while preserving the existing score behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100