Spellchecker don't find suggestion for concrete misspelled 6 letter words [LUCENE-4793]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 86
Description
Debugging Solr spellchecker (IndexBasedSpellchecker, delegating on lucene Spellchecker) behaviour i think i found a bug when the input is a 6 letter word:
- george
- anthem
- argued
- fluent
Due to the getMin() and getMax() the grams indexed for these terms are 3 and 4. So, the fields would be something like this:
- for "**george**"
- start3: "geo"
- start4: "geor"
- end3: "rge"
- end4: "orge"
- 3: "geo", "eor", "org", "rge"
- 4: "geor", "eorg", "orge"
- for "**anthem**"
- start3: "ant"
- start4: "anth"
- end3: "tem"
- end4: "them"
The problem shows up when the user swap 3rd a 4th characters, misspelling the word like this:
- geroge
- anhtem
The queries generated for this terms are: (SHOULD boolean queries)
- for "**geroge**"
- start3: "ger"
- start4: "gero"
- end3: "oge"
- end4: "roge"
- 3: "ger", "ero", "rog", "oge"
- 4: "gero", "erog", "roge"
- for "**anhtem**"
- start3: "anh"
- start4: "anht"
- end3: "tem"
- end4: "htem"
- 3: "anh", "nht", "hte", "tem"
- 4: "anht", "nhte", "htem"
So, as you can see, this kind of misspelling never matches the suitable suggestions although the edit distance is 0.95555556.
I think getMin(int l) and getMax(int l) should return 2 and 3, respectively, for l==6. Debugging other values i did not found any problem with any kind of misspelling.
---
Migrated from [LUCENE-4793](https://issues.apache.org/jira/browse/LUCENE-4793) by Samuel García Martínez (@samuelgmartinez)
Attachments: [SpellcheckerTest.java](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-4793/SpellcheckerTest.java)
Contributor guide
Research direction
Start with the attached SpellcheckerTest.java and trace the Lucene spellchecker code that implements getMin(int l) and getMax(int l). Reproduce the six-letter transposition examples such as “geroge” and “anhtem”, then add or update a regression test showing that suitable suggestions are found for these inputs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 40/100