apache / apache/lucene

UnifiedHighlighter: Convert PhraseHelper to use SpanCollector API [LUCENE-7578]

Open
#8,629 3 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major module:highlighter type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

The PhraseHelper of the UnifiedHighlighter currently collects position-spans per SpanQuery (and it knows which terms are in which SpanQuery), and then it filters PostingsEnum based on that. It's similar to how the original Highlighter WSTE works. The main problem with this approach is that it can be inaccurate for some nested span queries – #3363, #6518 (has the clearest example), #7854. Non-nested SpanQueries (e.g. that which is converted from a PhraseQuery or MultiPhraseQuery) are _not_ a problem.

---
Migrated from [LUCENE-7578](https://issues.apache.org/jira/browse/LUCENE-7578) by David Smiley (@dsmiley), 1 vote

Contributor guide

Open the contributing guide

Research direction

Start by locating PhraseHelper in the UnifiedHighlighter and reading the SpanCollector API, then review the nested SpanQuery cases described in issues #3363, #6518, and #7854. Done means PhraseHelper uses SpanCollector while preserving accurate handling of nested span queries and existing non-nested phrase queries.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.