apache / apache/lucene

FastVectorHighlighter Overlapping Proximity Queries Do Not Highlight [LUCENE-4734]

Open
#5,799 23 comments 0 reactions 0 assignees View on GitHub
affects-version:4.0 affects-version:4.1 affects-version:6.0 legacy-jira-fix-version:4.9 legacy-jira-fix-version:6.0 legacy-jira-label:FastVectorHighlighter legacy-jira-label:Highlighter legacy-jira-priority:Major module:highlighter type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

If a proximity phrase query overlaps with any other query term it will not be highlighted.

Example Text: A B C D E F G

Example Queries:

"B E"\~10 D
(D will be highlighted instead of "B C D E")

"B E"\~10 "C F"\~10
(nothing will be highlighted)

This can be traced to the FieldPhraseList constructor's inner while loop. From the first example query, the first TermInfo popped off the stack will be "B". The second TermInfo will be "D" which will not be found in the submap for "B E"\~10 and will trigger a failed match.

---
Migrated from [LUCENE-4734](https://issues.apache.org/jira/browse/LUCENE-4734) by Ryan Lauck, updated May 09 2016
Attachments: [lucene-4734.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-4734/lucene-4734.patch), [LUCENE-4734.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-4734/LUCENE-4734.patch), [LUCENE-4734-2.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-4734/LUCENE-4734-2.patch)
Linked issues:
- #5190
- #5190

Contributor guide

Open the contributing guide

Research direction

Start at the FieldPhraseList constructor, especially its inner while loop, and trace how overlapping proximity queries are matched against successive TermInfo entries. Use the two query examples as regression cases: the first should highlight “B C D E,” and the second should highlight the overlapping phrase rather than nothing.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.