apache / apache/lucene

Add tokenized version of o.o. to Stempel stopwords [LUCENE-8416]

Open
#9,462 1 comment 0 reactions 0 assignees View on GitHub
legacy-jira-label:easyfix legacy-jira-label:newbie legacy-jira-priority:Trivial module:analysis type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

The Stempel stopword list ( lucene-solr/lucene/analysis/stempel/src/resources/org/apache/lucene/analysis/pl/stopwords.txt ) contains "o.o." which is a good stopword (it's part of the abbreviation for "limited liability company", which is "[sp. z o.o.](https://en.wiktionary.org/wiki/sp._z_o.o.)". However, the standard tokenizer changes "o.o." to "o.o" so the stopword filter has no effect.

Add "o.o" to the stopword list. (It's probably okay to leave "o.o." in the list, though, in case a different tokenizer is used.)

---
Migrated from [LUCENE-8416](https://issues.apache.org/jira/browse/LUCENE-8416) by Trey Jones, updated Sep 12 2018
Pull requests: https://github.com/apache/lucene-solr/pull/447

Contributor guide

Open the contributing guide

Research direction

Open lucene-solr/lucene/analysis/stempel/src/resources/org/apache/lucene/analysis/pl/stopwords.txt and inspect the existing "o.o." entry alongside the Standard Tokenizer behavior described in the issue. Done means the tokenized "o.o" form is present so the stopword filter can match it, while the original entry may remain.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.