consider unifying stopwords formats for 5.0 [LUCENE-5227]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Hossman has background on #6275.
The story is we added these to lucene (it used to be a 'svn export' from snowball tree!!!!) and i had several reasons for supporting the snowball format:
1. svn export/easier to maintain diffs
2. in lucene from Analyzer APIs, as a default "Set" it didnt much matter.
3. the snowball format is nice the way they present e.g. inflection tables for some languages that inflect pronouns.
But the reality is:
1. people try to use these from e.g. solr and hit traps.
2. these things are not changing hardly at all in the snowball repository.
3. we don't do svn export anymore.
4. the "tables" could just be preserved inside # comments and still explain why the words are in the file.
We could convert our files for 5.0, and just update our stuff appropriately, and of course still support parsing the old format, and it wouldnt break anyway, just reduce traps i think.
---
Migrated from [LUCENE-5227](https://issues.apache.org/jira/browse/LUCENE-5227) by Robert Muir (@rmuir), updated May 09 2016
Contributor guide
Research direction
Start with the background in issue #6275 and compare the current Snowball and Lucene stopword formats mentioned here. Done means converting the stopword files for 5.0, updating related Lucene usage, and retaining support for parsing the old format; the issue does not name specific files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100