apache / apache/lucene

WordBreakSpellChecker: suggest in order of minimum term frequency [LUCENE-6237]

Open
#7,299 0 comments 0 reactions 0 assignees View on GitHub
affects-version:4.10.3 legacy-jira-priority:Minor module:suggest type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

When the Word-Break Spellchecker breaks a term into more than once, currently the results are ordered first by the number of breaks, then by the _maximum_ frequency of the new terms. But in many cases, it seems better to sort secondarily on the _minimum_ frequency.

See this thread from the users' list: http://lucene.472066.n3.nabble.com/alternativeTermCount-and-WordBreakSolrSpellChecker-combination-not-working-tp4185352p4185489.html

As a follow-on, we should expose this new configuration detail for solr users, possibly making it the default.

---
Migrated from [LUCENE-6237](https://issues.apache.org/jira/browse/LUCENE-6237) by James Dyer (@jdyer1), 1 vote

Contributor guide

Open the contributing guide

Research direction

Start by locating the WordBreakSpellChecker implementation and the Solr spellchecker configuration path, then review the linked users-list discussion for the intended ordering. Done means split-term suggestions use minimum term frequency as the secondary ordering and the Solr configuration behavior is explicitly decided and covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.