apache / apache/lucene

BlockMaxConjunctionScorer should compute better lower bounds of the required scores [LUCENE-8177]

Open
#9,225 0 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

Assuming N scorers, `BlockMaxConjunctionScorer` computes a lower bound of the sum of scores for scorers 0..i, for any given 0 <= i < N.

For instance say you are searching for "quick AND fox", that a hit needs a score of 4 to be competitive and that "quick" contributes at most 3 to the score and "fox" 2. This means that for a given hit to be competitive, the sum of scores must be at least 4-maxScore(fox)=4-2=2 after having scored "quick" and 4 after having scored "fox".

Currently we have this in BlockMaxConjunctionScorer:
{code:java}
// Also compute the minimum required scores for a hit to be competitive
// A double that is less than 'score' might still be converted to 'score'
// when casted to a float, so we go to the previous float to avoid this issue
minScores[minScores.length - 1] = minScore > 0 ? Math.nextDown(minScore)
{code}

We currently use `Math.minDown(float)` to be safe, but we would get a better bound by computing the lowest double that is converted to `minScore` when casted to a float.

---
Migrated from [LUCENE-8177](https://issues.apache.org/jira/browse/LUCENE-8177) by Adrien Grand (@jpountz)

Contributor guide

Open the contributing guide

Research direction

Start by locating BlockMaxConjunctionScorer and reading its current minScores calculation. Reproduce how double values are converted to float around minScore, then determine whether the lower bound can be tightened as described. Done means the scorer computes the more precise lower bound without changing the intended competitive-hit behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.