Exceptions handling in methods of SegmentingTokenizerBase [LUCENE-9588]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
The current interface of **setNextSentence** and **incrementWord** methods in **SegmentingTokenizerBase** do not define the checked exceptions, which makes it troublesome to be inherited.
For example, if we override the incrementWord with a logic that invoke incrementToken on another tokenizer, the incrementToken raises the IOException but the incrementWord is not defined to handle it.
I think having setNextSentence and incrementWord handle the IOException would make the SegmentingTokenizerBase easier to be used.
---
Migrated from [LUCENE-9588](https://issues.apache.org/jira/browse/LUCENE-9588) by Nguyen Minh Gia Huy, updated Nov 05 2020
Contributor guide
Research direction
Start at SegmentingTokenizerBase and inspect the setNextSentence and incrementWord method signatures, then trace the incrementToken call mentioned in the issue. The work is done when these methods can handle the IOException needed by inherited implementations without breaking the tokenizer API; verify the affected Java tests or compilation afterward.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100