CJKAnalyzer not matching mutlibyte character followed by non-multibyte character [LUCENE-2673]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Here is a listing of text indexed in a field, followed by various search terms that did or did not match the document.
[QES様文字化けテスト]
QES -> retrievable
QES様 -> not retrievable
QES様文字化けテスト -> retrievable
[SOA基盤]
SOA ->retrievable
SOA基 -> not retrievable
SOA基盤 -> retrievable
[日経BP]
日経 -> retrievable
日経B -> not retrievable
日経BP -> retrievable
---
Migrated from [LUCENE-2673](https://issues.apache.org/jira/browse/LUCENE-2673) by Kevin Hayen, 1 vote, updated May 16 2011
Contributor guide
Research direction
Start by reading the CJKAnalyzer implementation and its analyzer tests, then reproduce the reported queries where a multibyte character is followed by an ASCII character. Done means the listed mixed-character searches match consistently with the surrounding full-term searches, with a regression test covering the examples.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100