apache / apache/lucene

Can we support vectors to be loaded with direct I/O for full precision re-ranking?

Open
#14,746 1 comment 0 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Spin-off from discussion in https://github.com/apache/lucene/pull/14708. One of the concern with with full precision (FP) re-ranking (for quantized vectors) is that if we use off-heap vector reader it will page-in the FP vector data and can compete with quantized vector data which are used for HNSW graph search. As HNSW will suffer the performance greatly if the vectors are not in memory, for instance with limited memory, can we support a mode to let the FP vectors be loaded with direct I/O? (Or if this is already possible?)

For integrating with the existing quantized vectors codec, is my understanding correct that we will need to create a new codec/vector reader that extend from the [existing reader](https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/codecs/lucene102/Lucene102BinaryQuantizedVectorsFormat.java#L126C16-L126C31) and use a different raw vector format?

I can try this, but wondering what the community think about it. Is there other use case that needs a on-heap direct I/O vector readers as well?

Contributor guide

Open the contributing guide

Research direction

Start with the existing reader in lucene/core/src/main/java/org/apache/lucene/codecs/lucene102/Lucene102BinaryQuantizedVectorsFormat.java and review the discussion in PR 14708. Determine how a direct-I/O mode for full-precision vectors would integrate with the quantized vectors codec and whether a new vector reader or raw vector format is required. Done means a concrete, agreed design covering the mode and relevant use cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.