Should we move vector bulk-scoring APIs to VectorUtilSupport?
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
Off-heap float vector bulk-scoring was introduced in #14980, and showed some nice performance improvements! (vectors are scored in chunks of a specific machine-dependent size to leverage SIMD instructions, and if I understand correctly, this optimization amortizes the cost of loading chunks of the query vector _across N documents_, instead of loading it _once per document_).
There is currently an open issue to enable off-heap bulk-scoring for other vectors (byte vectors and quantized versions): #15155.
I wonder if we should move the bulk-scoring APIs one level lower (from [`RandomVectorScorer`](https://github.com/apache/lucene/blob/2822b28b5ab042672d6cb569589128e5dcafa648/lucene/core/src/java/org/apache/lucene/util/hnsw/RandomVectorScorer.java#L37-L54) to [`VectorUtilSupport`](https://github.com/apache/lucene/blob/2822b28b5ab042672d6cb569589128e5dcafa648/lucene/core/src/java/org/apache/lucene/internal/vectorization/VectorUtilSupport.java#L25)) so that the native vectorization provider (#15508) can also benefit from it? (which I think was added to make use of specific instructions that the JVM cannot compile to)
Contributor guide
Research direction
Read RandomVectorScorer.java and VectorUtilSupport.java first, then review the context in issues #14980, #15155, and #15508. Determine whether moving bulk-scoring APIs enables both off-heap and native vectorization providers without an unsuitable abstraction. Done means a documented design decision and, if accepted, a coherent API change with affected vector implementations updated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- performance, search
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100