apache / apache/lucene

Switch BpVectorReorderer to use bulk scoring

Open
#15,219 4 comments 0 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Reading through the code of the BpVectorReorderer, I noticed that it simply reads the vectors on heap and then does individual comparisons between the centroids and the vectors used in the threads.

I think it would be much faster to bulk score against the left and right centroids instead. I am not 100% sure if this is directly available over a merged view (that might be step one...).

I am particularly looking at the `computeBias` method. Seems like the caller in the `computeBiasTask` could create a leftScorer and a rightScorer and bulk score over each centroid.

Contributor guide

Open the contributing guide

Research direction

Start with BpVectorReorderer, especially computeBias and its computeBiasTask caller. Investigate whether bulk scoring is available over the merged view, then determine how left and right scorers would be used for the centroid comparisons. Done means the reorderer uses bulk scoring with equivalent results and the relevant existing checks still pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.