apache / apache/lucene

SpanishLightStemmer stemming errors [LUCENE-7420]

Open
#8,473 3 comments 0 reactions 0 assignees View on GitHub
legacy-jira-label:analysis legacy-jira-label:stemming legacy-jira-priority:Minor module:analysis type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

SpanishLightStemmer only applies stemming if the original word is 5 characters or longer. It tries to removing gender and numbers, removing final vowel and s if they exist.

So, perro, perra, perros and perras (dog, bitch, dogs and bitches) all become 'perr'.
The problem arises with shorter words. Gatos and Gatas (cats / female cats) all become 'gat', but the singular forms (gato, gata) are below the 5 character threshold, so they wouldn't be stemmed.

---
Migrated from [LUCENE-7420](https://issues.apache.org/jira/browse/LUCENE-7420) by J Pardos, updated Mar 07 2018
Attachments: [SpanishLightStemmer.java.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-7420/SpanishLightStemmer.java.patch)

Contributor guide

Open the contributing guide

Research direction

Locate the SpanishLightStemmer implementation and review the attached SpanishLightStemmer.java.patch first. Check the length threshold and existing gender, number, and final-vowel handling using the gato/gata and gatos/gatas examples. Done means singular and plural forms stem consistently without regressing the perro examples.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.