ShingleFilter should handle positionIncrement of zero, e.g. synonyms [LUCENE-3475]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
ShingleFilter is creating shingles for a single term that has been expanded by synonyms when it shouldn't. The position increment is 0.
As an example, I have an Analyzer with a SynonymFilter followed by a ShingleFilter. Assuming car and auto are synonyms, the SynonymFilter produces two tokens and position 1: car, auto. The ShingleFilter is then producing 3 tokens, when there should only be two: car, car auto, auto. This behavior seems incorrect.
---
Migrated from [LUCENE-3475](https://issues.apache.org/jira/browse/LUCENE-3475) by Cameron, 4 votes, updated Mar 13 2018
Attachments: [LUCENE-3475.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-3475/LUCENE-3475.patch) (versions: 2)
Linked issues:
- #5242
Contributor guide
Assessment
This issue has not been assessed yet.