add simple japanese tokenizer, based on tinysegmenter [LUCENE-2522]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
TinySegmenter (http://www.chasen.org/\~taku/software/TinySegmenter/) is a tiny japanese segmenter.
It was ported to java/lucene by Kohei TAKETA <k-tak@void.in>,
and is under friendly license terms (BSD, some files explicitly disclaim copyright to the source code, giving a blessing instead)
Koji knows the author, and already contacted about incorporating into lucene:
```
I've contacted Takeda-san who is the creater of Java version of
TinySegmenter. He said he is happy if his program is part of Lucene.
He is a co-author of my book about Solr published in Japan, BTW. ;-)
```
---
Migrated from [LUCENE-2522](https://issues.apache.org/jira/browse/LUCENE-2522) by Robert Muir (@rmuir), updated May 09 2016
Attachments: [LUCENE-2522.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-2522/LUCENE-2522.patch) (versions: 3)
Contributor guide
Assessment
This issue has not been assessed yet.