apache / apache/lucene

Should we move vector bulk-scoring APIs to VectorUtilSupport?

Open
#16,556 4 comments 0 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Off-heap float vector bulk-scoring was introduced in #14980, and showed some nice performance improvements! (vectors are scored in chunks of a specific machine-dependent size to leverage SIMD instructions, and if I understand correctly, this optimization amortizes the cost of loading chunks of the query vector _across N documents_, instead of loading it _once per document_).

There is currently an open issue to enable off-heap bulk-scoring for other vectors (byte vectors and quantized versions): #15155.

I wonder if we should move the bulk-scoring APIs one level lower (from [`RandomVectorScorer`](https://github.com/apache/lucene/blob/2822b28b5ab042672d6cb569589128e5dcafa648/lucene/core/src/java/org/apache/lucene/util/hnsw/RandomVectorScorer.java#L37-L54) to [`VectorUtilSupport`](https://github.com/apache/lucene/blob/2822b28b5ab042672d6cb569589128e5dcafa648/lucene/core/src/java/org/apache/lucene/internal/vectorization/VectorUtilSupport.java#L25)) so that the native vectorization provider (#15508) can also benefit from it? (which I think was added to make use of specific instructions that the JVM cannot compile to)

Contributor guide

Open the contributing guide

Research direction

Read RandomVectorScorer.java and VectorUtilSupport.java first, then review the context in issues #14980, #15155, and #15508. Determine whether moving bulk-scoring APIs enables both off-heap and native vectorization providers without an unsuitable abstraction. Done means a documented design decision and, if accepted, a coherent API change with affected vector implementations updated.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance, search
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.