Allow DirectSpellchecker to use totalTermFrequency rather than docFrequency [LUCENE-4813]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
we have a bunch of new statistics in on our term dictionaries that we should make use of where it makes sense. For DirectSpellChecker totalTermFreq and sumTotalTermFreq might be better suited for spell correction on top of a fulltext index than docFreq and maxDoc
---
Migrated from [LUCENE-4813](https://issues.apache.org/jira/browse/LUCENE-4813) by Simon Willnauer (@s1monw), updated May 09 2016
Attachments: [LUCENE-4813.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-4813/LUCENE-4813.patch) (versions: 2)
Contributor guide
Research direction
Start with DirectSpellChecker and the attached LUCENE-4813.patch, then compare the proposed use of totalTermFreq and sumTotalTermFreq with the current docFreq and maxDoc statistics. Done means spell correction can use the newer term-frequency statistics where appropriate, with behavior validated against the relevant Lucene tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100