apache / apache/lucene

BooleanWeight should decide how to execute minNrShouldMatch [LUCENE-4872]

Open
#5,937 11 comments 0 reactions 0 assignees View on GitHub
legacy-jira-fix-version:4.9 legacy-jira-fix-version:6.0 legacy-jira-priority:Major module:core/search type:task
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

#5637 adds a dedicated document-at-time scorer for minNrShouldMatch which can use advance() behind the scenes.

In cases where you have some really common terms and some rare ones this can be a huge performance improvement.

On the other hand BooleanScorer might still be faster in some cases.

We should think about what the logic should be here: one simple thing to do is to always use the new scorer when minShouldMatch is set: thats where i'm leaning.

But maybe we could have a smarter heuristic too, perhaps based on cost()

---
Migrated from [LUCENE-4872](https://issues.apache.org/jira/browse/LUCENE-4872) by Robert Muir (@rmuir), 1 vote, updated May 09 2016
Parent: #5637
Attachments: [crazyMinShouldMatch.tasks](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-4872/crazyMinShouldMatch.tasks)

Contributor guide

Open the contributing guide

Research direction

Start with the dedicated document-at-a-time scorer from #5637 and inspect how BooleanWeight currently chooses between it and BooleanScorer for minNrShouldMatch. Review the possible cost() heuristic and the crazyMinShouldMatch.tasks attachment, then define and validate a selection policy that is faster for both common and rare-term cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.