Enhance WordDelimiterFilter to skip operation on tokens marked as keywords. [LUCENE-6903]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
We have a github pull request for a feature to allow WDF to skip operation when the keyword marker filter has marked tokens.
https://github.com/apache/lucene-solr/pull/210
This will also add a new configuration parameter called "splitKeywordTokens" that will default to false in 6.0 and true in earlier versions.
---
Migrated from [LUCENE-6903](https://issues.apache.org/jira/browse/LUCENE-6903) by Shawn Heisey (@elyograg)
Attachments: [LUCENE-6903.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-6903/LUCENE-6903.patch)
Contributor guide
Research direction
Review pull request 210 and the attached LUCENE-6903.patch first, since the proposed implementation is already linked from the issue. Confirm that WordDelimiterFilter skips marked keyword tokens and that the splitKeywordTokens parameter has the stated defaults for Lucene 6.0 and earlier versions; the existing pull request should show the relevant tests and files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100