SnowballFilter to support minTokenLength [LUCENE-5208]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
In some cases you don't want the stemmer to consider short tokens. Instead of modifying the snowball code, testing it, compiling it to Java code and the whole hassle, with this patch you can set a minTokenLength.
---
Migrated from [LUCENE-5208](https://issues.apache.org/jira/browse/LUCENE-5208) by Markus Jelsma, updated May 09 2016
Attachments: [LUCENE-5208-trunk.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-5208/LUCENE-5208-trunk.patch)
Contributor guide
Research direction
Start at the SnowballFilter entry point and review the attached LUCENE-5208-trunk.patch to understand the proposed scope. Run the existing SnowballFilter tests, then verify that short tokens can be excluded from stemming through a configurable minimum token length.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100