apache / apache/lucene

HunspellStemmer dedup not fully functional [LUCENE-3670]

Open
#4,744 0 comments 0 reactions 0 assignees View on GitHub
affects-version:3.5 legacy-jira-priority:Minor module:analysis type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

public List<Stem> uniqueStems(char word[], int length) assumes that word.length is always same as length and in cases where it is not, duplicate stems are returned.
Quick patch may be applied to trim extra characters:

if (word.length > length) {
terms.add(Arrays.copyOf(word, length));
} else {
terms.add(word);
}

---
Migrated from [LUCENE-3670](https://issues.apache.org/jira/browse/LUCENE-3670) by Emir Arnautovic

Contributor guide

Open the contributing guide

Research direction

Locate HunspellStemmer.uniqueStems(char[], int) in the Java source and inspect how the supplied length is used when collecting stems. Reproduce the case where the array is longer than length, then verify the change prevents duplicate stems while preserving the requested word length.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.