WordDelimiterFilter stemEnglishPossessive option does not work for Tokens ending with a digit [LUCENE-5203]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
For Token ending with "{digit}'s" the WordDelimiterFilter ignores the stemEnglishPossessive configuration
e.g. in the text "SD500's poor underwater performance" the "'s" will not be stemmed from SD500 even if stemEnglishPossessive="1".
The reason is that "WordDelimiterIterator#endsWithPossessive(int pos)" checks for
isAlpha(charType(text[pos - 3]))
while it should also allow for isDigits(int type).
---
Migrated from [LUCENE-5203](https://issues.apache.org/jira/browse/LUCENE-5203) by Rupert Westenthaler, 1 vote
Contributor guide
Research direction
Start at WordDelimiterIterator#endsWithPossessive(int pos) and inspect how the character type before "'s" is checked. Confirm that tokens ending in a digit followed by "'s" are handled when stemEnglishPossessive is enabled, and verify the existing WordDelimiterFilter behavior for the reported SD500's example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100