apache / apache/lucene

[nori] A word that is registered in advance, but the words are not separated and recognized as 'UNKNOWN' [LUCENE-8772]

Open
#9,817 2 comments 0 reactions 0 assignees View on GitHub
affects-version:7.5 affects-version:7.6 affects-version:7.7 affects-version:7.7.1 affects-version:8.0 legacy-jira-priority:Major module:analysis type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

hello,

In case of 'nori', if there is no word starting from the left, 'UNKNOWN' is analyzed even if there is a word already registered in the middle.
So here is the question.
Does nori analyze only on the left side and do not analyze from the right side?
Could this be solved?

 

ex)

input => 갊수학

Condition

dictionary registered : 수학
dictionary Unregistered : 갊

result => 갊수학

![image-2019-04-19-11-32-56-310.png](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-8772/image-2019-04-19-11-32-56-310.png)

---
Migrated from [LUCENE-8772](https://issues.apache.org/jira/browse/LUCENE-8772) by YOO JEONGIN, updated Apr 22 2019
Attachments: [image-2019-04-19-11-32-56-310.png](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-8772/image-2019-04-19-11-32-56-310.png)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the nori example with input "갊수학", with "수학" registered and "갊" unregistered. Trace how nori analyzes the input and determine whether the observed UNKNOWN result matches the intended behavior. Done means the behavior is clarified and, if confirmed as a bug, covered by a regression test and corrected.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.