FastVectorHighlighter ignores position in PhraseQuery [LUCENE-7551]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
The PhraseQuery contains a list of terms and the positions for each term. The FVH ignores the term position and assumes that a phrase query is always dense. As a result phrase query with gaps are not highlighted at all. This is problematic for text fields that use a FilteringTokenFilter. This token filter removes tokens but preserves the position increment of each removal.
Bottom line is that using this token filter breaks the highlighting of phrase query that contains filtered tokens.
---
Migrated from [LUCENE-7551](https://issues.apache.org/jira/browse/LUCENE-7551) by Jim Ferenczi (@jimczi), 1 vote
Linked issues:
- [SOLR-3724](https://issues.apache.org/jira/browse/SOLR-3724)
- #8603
Contributor guide
Research direction
Start by tracing the FastVectorHighlighter handling of PhraseQuery term positions, then reproduce the case with a FilteringTokenFilter that removes tokens while preserving position increments. Done means phrase queries containing filtered-token gaps are highlighted correctly rather than being omitted.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100