Combine Nori and Kuromoji DictionaryBuilder [LUCENE-8817]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
This issue is related to #9860.
Currently Nori and Kuromoji Analyzer use the same dictionary structure. (MeCab)
If we make combine DictionaryBuilder, we can reduce the code size.
But this task may have a dependency on the language.
(like HEADER string in BinaryDictionary and CharacterDefinition, methods in BinaryDictionaryWriter, ...)
On the other hand, there are many overlapped classes.
The purpose of this patch is to provide users of Nori and Kuromoji with the same system dictionary generator.
It may take some time because there is a little workload.
The work will be based on the latest master, and if the #9860 is finished first, I will pull the latest code and proceed.
---
Migrated from [LUCENE-8817](https://issues.apache.org/jira/browse/LUCENE-8817) by Namgyu Kim (@danmuzi), updated Jun 10 2019
Linked issues:
- #9860
Contributor guide
Research direction
Compare the Nori and Kuromoji DictionaryBuilder implementations and their shared MeCab dictionary structure. Start with BinaryDictionary, CharacterDefinition, and BinaryDictionaryWriter, while checking the dependency on #9860 and the latest master. Done means both analyzers use one system dictionary generator without losing their language-specific behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100