Optimize non-scoring minimum-should-match queries that fall back to WANDScorer
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
BooleanScorerSupplier#opt` currently selects `WANDScorer` whenever `minShouldMatch > 1`, including score modes that don't require scores:
https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/search/BooleanScorerSupplier.java#L580
For these non-scoring modes, `WANDScorer` cannot benefit from block-max score pruning. It returns candidate documents through its approximation and performs the actual minimum-should-match verification in
`TwoPhaseIterator.matches()`.
Before https://github.com/apache/lucene-solr/pull/2205, query used `MinShouldMatchSumScorer`, whose `nextDoc()` implementation searched internally for a document satisfying `minShouldMatch`. Although the two implementations are similar, this
behavior may be more efficient when scores are not required.
In our production environment, upgrading lucene make query pattern from `MinShouldMatchSumScorer` to `WANDScorer` caused more CPU consumed.
I plan to benchmark `MinShouldMatchSumScorer` and `WANDScorer` with non-scoring query to see the difference.
Contributor guide
Research direction
Start in lucene/core/src/java/org/apache/lucene/search/BooleanScorerSupplier.java at the BooleanScorerSupplier#opt selection around line 580. Compare MinShouldMatchSumScorer and WANDScorer for minShouldMatch > 1 in non-scoring modes, focusing on their candidate and minimum-should-match verification behavior. Benchmark both implementations; done means establishing the performance difference and whether the scorer choice should change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100