apache / apache/lucene

NPE on wildcard-based overlapping intervals highlighting [LUCENE-10075]

Open
#11,113 3 comments 0 reactions 0 assignees View on GitHub
affects-version:8.7 affects-version:9.0 legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

UnifiedHighlighter with WEIGHT_MATCHES flag throws an NullPointerException on overlapping intervals with wildcard term.

Minimal reproducible example

Doc: "Compare Computer Science"

Query: Intervals.maxgaps(1, Intervals.ordered(Intervals.wildcard(new BytesRef("comp\*")), Intervals.term("science")));

Stacktrace:

 

```java
java.lang.NullPointerException: Cannot invoke "org.apache.lucene.search.MatchesIterator.endPosition()" because the return value of "org.apache.lucene.util.PriorityQueue.top()" is nulljava.lang.NullPointerException: Cannot invoke "org.apache.lucene.search.MatchesIterator.endPosition()" because the return value of "org.apache.lucene.util.PriorityQueue.top()" is null
at org.apache.lucene.search.DisjunctionMatchesIterator.endPosition(DisjunctionMatchesIterator.java:233) at org.apache.lucene.queries.intervals.MultiTermIntervalsSource$1.endPosition(MultiTermIntervalsSource.java:132) at org.apache.lucene.search.FilterMatchesIterator.endPosition(FilterMatchesIterator.java:49) at org.apache.lucene.queries.intervals.CachingMatchesIterator.getSubMatches(CachingMatchesIterator.java:88) at org.apache.lucene.queries.intervals.MinimizingConjunctionMatchesIterator.getSubMatches(MinimizingConjunctionMatchesIterator.java:96) at org.apache.lucene.queries.intervals.IntervalMatches$1.getSubMatches(IntervalMatches.java:82) at org.apache.lucene.search.FilterMatchesIterator.getSubMatches(FilterMatchesIterator.java:64) at org.apache.lucene.search.uhighlight.OffsetsEnum$OfMatchesIteratorWithSubs.nextWhenMatchesIterator(OffsetsEnum.java:209) at org.apache.lucene.search.uhighlight.OffsetsEnum$OfMatchesIteratorWithSubs.nextPosition(OffsetsEnum.java:201) at org.apache.lucene.search.uhighlight.FieldHighlighter.highlightOffsetsEnums(FieldHighlighter.java:134) at org.apache.lucene.search.uhighlight.FieldHighlighter.highlightFieldForDoc(FieldHighlighter.java:83) at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFieldsAsObjects(UnifiedHighlighter.java:635) at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFields(UnifiedHighlighter.java:505) at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlightFields(UnifiedHighlighter.java:483) at org.apache.lucene.search.uhighlight.UnifiedHighlighter.highlight(UnifiedHighlighter.java:416)
```

 

Search by the same query completes without any exception, ordered/unordered and larger gaps have no effect.

---
Migrated from [LUCENE-10075](https://issues.apache.org/jira/browse/LUCENE-10075) by Nikolay Khitrin (@khitrin), updated Oct 04 2021
Pull requests: https://github.com/apache/lucene/pull/270

Contributor guide

Open the contributing guide

Research direction

Start with the minimal UnifiedHighlighter reproduction and trace the stack through DisjunctionMatchesIterator.java, MultiTermIntervalsSource.java, and the interval matching classes named in the trace. Confirm the failure with the wildcard query and add a regression test showing that highlighting overlapping intervals completes without an exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.