NPE on wildcard-based overlapping intervals highlighting [LUCENE-10075]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
UnifiedHighlighter with WEIGHT_MATCHES flag throws an NullPointerException on overlapping intervals with wildcard term.
Minimal reproducible example
Doc: "Compare Computer Science"
Query: Intervals.maxgaps(1, Intervals.ordered(Intervals.wildcard(new BytesRef("comp\*")), Intervals.term("science")));
Stacktrace:
```java
java.lang.NullPointerException: Cannot invoke "org.apache.lucene.search.MatchesIterator.endPosition()" because the return value of "org.apache.lucene.util.PriorityQueue.top()" is nulljava.lang.NullPointerException: Cannot invoke "org.apache.lucene.search.MatchesIterator.endPosition()" because the return value of "org.apache.lucene.util.PriorityQueue.top()" is null
at org.apache.lucene.search.DisjunctionMatchesIterator.endPosition(DisjunctionMatchesIterator.java:233) at org.apache.lucene.queries.intervals.MultiTermIntervalsSource$1.endPosition(MultiTermIntervalsSource.java:132) at org.apache.lucene.search.FilterMatchesIterator.endPosition(FilterMatchesIterator.java:49) at org.apache.lucene.queries.intervals.CachingMatchesIterator.getSubMatches(CachingMatchesIterator.java:88) at org.apache.lucene.queries.intervals.MinimizingConjunctionMatchesIterator.getSubMatches(MinimizingConjunctionMatchesIterator.java:96) at org.apache.lucene.queries.intervals.IntervalMatches$1.getSubMatches(IntervalMatches.java:82) at org.apache.lucene.search.FilterMatchesIterator.getSubMatches(FilterMatchesIterator.java:64) at org.apache.lucene.search.uhighlight.OffsetsEnum$OfMatchesIteratorWithSubs.nextWhenMatchesIterator(OffsetsEnum.java:209) at org.apache.lucene.search.uhighlight.OffsetsEnum$OfMatchesIteratorWithSubs.nextPosition(OffsetsEnum.java:201) at org.apache.lucene.search.uhighlight.FieldHighlighter.highlightOffsetsEnums(FieldHighlighter.java:134) at org.apache.lucene.search.uhighlight.FieldHighlighter.highlightFieldForDoc(FieldHighlighter.java:83) at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFieldsAsObjects(UnifiedHighlighter.java:635) at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFields(UnifiedHighlighter.java:505) at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFields(UnifiedHighlighter.java:483) at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlight(UnifiedHighlighter.java:416)
```
Search by the same query completes without any exception, ordered/unordered and larger gaps have no effect.
---
Migrated from [LUCENE-10075](https://issues.apache.org/jira/browse/LUCENE-10075) by Nikolay Khitrin (@khitrin), updated Oct 04 2021
Pull requests: https://github.com/apache/lucene/pull/270
Contributor guide
Research direction
Start with the minimal UnifiedHighlighter reproduction and trace the stack through DisjunctionMatchesIterator.java, MultiTermIntervalsSource.java, and the interval matching classes named in the trace. Confirm the failure with the wildcard query and add a regression test showing that highlighting overlapping intervals completes without an exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100