UnifiedHighlighter not highlighting all terms relevant in SpanNearQuery [LUCENE-7682]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Original text: "Something for protecting wildlife feed in a feed thing."
Query is:
SpanNearQuery with Slop 9 - in order -
1. SpanTermQuery(wildlife)
2. SpanTermQuery(feed)
This should highlight both instances of "feed" since they are both within slop of 9 of "wildlife". However, only the first instance is highlighted. This occurs with unordered SpanNearQuery as well. Test below replicates. Affects both the current 6.x line and master.
Test that fits within TestUnifiedHighlighterMTQ:
```Java
public void testOrderedSpanNearQueryWithDupeTerms() throws Exception {
RandomIndexWriter iw = new RandomIndexWriter(random(), dir, indexAnalyzer);
Document doc = new Document();
doc.add(new Field("body", "Something for protecting wildlife feed in a feed thing.", fieldType));
doc.add(newTextField("id", "id", Field.Store.YES));
iw.addDocument(doc);
IndexReader ir = iw.getReader();
iw.close();
IndexSearcher searcher = newSearcher(ir);
UnifiedHighlighter highlighter = new UnifiedHighlighter(searcher, indexAnalyzer);
int docID = searcher.search(new TermQuery(new Term("id", "id")), 1).scoreDocs[0].doc;
SpanTermQuery termOne = new SpanTermQuery(new Term("body", "wildlife"));
SpanTermQuery termTwo = new SpanTermQuery(new Term("body", "feed"));
SpanNearQuery topQuery = new SpanNearQuery.Builder("body", true)
.setSlop(9)
.addClause(termOne)
.addClause(termTwo)
.build();
int[] docIds = new int[] {docID};
String snippets[] = highlighter.highlightFields(new String[] {"body"}, topQuery, docIds, new int[] {2}).get("body");
assertEquals(1, snippets.length);
assertEquals("Something for protecting wildlife feed in a feed thing.", snippets[0]);
ir.close();
}
```
---
Migrated from [LUCENE-7682](https://issues.apache.org/jira/browse/LUCENE-7682) by Michael Braun (@michaelbraun), updated Aug 16 2017
Linked issues:
- #7595
Contributor guide
Research direction
Start by locating and running the TestUnifiedHighlighterMTQ test mentioned in the issue, especially the provided ordered SpanNearQuery case. Trace the UnifiedHighlighter behavior for duplicate terms within the span query. Done means the test passes and both occurrences of “feed” are highlighted in the expected snippet.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100