apache / apache/lucene

split off the spell check word and surface form in spell check dictionary [LUCENE-3888]

Open
#4,961 17 comments 0 reactions 1 assignee Claimed by @kojisekig View on GitHub
legacy-jira-fix-version:4.9 legacy-jira-fix-version:6.0 legacy-jira-priority:Minor module:suggest type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

The "did you mean?" feature by using Lucene's spell checker cannot work well for Japanese environment unfortunately and is the longstanding problem, because the logic needs comparatively long text to check spells, but for some languages (e.g. Japanese), most words are too short to use the spell checker.

I think, for at least Japanese, the things can be improved if we split off the spell check word and surface form in the spell check dictionary. Then we can use ReadingAttribute for spell checking but CharTermAttribute for suggesting, for example.

---
Migrated from [LUCENE-3888](https://issues.apache.org/jira/browse/LUCENE-3888) by Koji Sekiguchi (@kojisekig), 2 votes, updated May 09 2016
Attachments: [LUCENE-3888.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-3888/LUCENE-3888.patch) (versions: 6)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.