apache / apache/lucene

Removing grave accent in Dutch Snowball algorithm [LUCENE-9295]

Open
#10,335 1 comment 0 reactions 0 assignees View on GitHub
legacy-jira-label:snowball legacy-jira-priority:Minor type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

I have a concern on how [Dutch Snowball algorithm](http://snowball.tartarus.org/algorithms/dutch/stemmer.html) handles the grave accent of **è.**

It removes the grave accents on **`à`**, `*ò*`, **`ù`**, **`ì`** but doesn't with **è**. I wonder if there is something special with **è** that the stemmer wants to ignore it.

Also, from I found out that grave accent is not used commonly in Dutch anymore except in some borrowed French words.

If **è** is not that common in Dutch, removing grave accent on it sounds reasonable to me and definitely benefits search recall in general.

I would like to know if anyone had a strong opinion on this topic ? It would be also nice if you have some point of views as a  Dutch speaker.

Thanks !

---
Migrated from [LUCENE-9295](https://issues.apache.org/jira/browse/LUCENE-9295) by Nguyen Minh Gia Huy, updated Jul 23 2020

Contributor guide

Open the contributing guide

Research direction

No Lucene source file or test is named. Start with the linked Dutch Snowball algorithm and locate the Lucene Dutch stemmer implementation, then compare its handling of è with à, ò, ù, and ì. Done means the project has a decided, tested behavior for grave-accent removal on è.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.