apache / apache/lucene

Examine adding more off-heap vector scoring

Open
#13,515 2 comments 4 reactions 1 assignee Claimed by @ChrisHegarty View on GitHub
vector-based-search
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

This issue has been filed to give consideration to the possibility of adding more off-heap vector scorers.

We recently added a MemorySegment Vector scorer for scoring byte vectors without copying on-heap - scorer loads values directly from the backing memory segment when available. Benchmarks show an approx 2x performance improvement by avoiding copying the vector data on-heap.

#### Floats

Loading vector data from float[] (the fallback), is only supported in JDK 22 - which might be fine. See https://bugs.openjdk.org/browse/JDK-8318678

#### Scalar quantized

Each of the possible quantized vector element sizes can be evaluated separately.
* int7
* int4
* 1 bit ?

relates #13339

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.