apache / apache/lucene

Move vector search from IndexInput to RandomAccessInput

Open
#13,938 7 comments 0 reactions 0 assignees View on GitHub
type:task vector-based-search
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Vector search currently loads vectors from disk by issuing a `seek()` followed by a `readFloats()`. We should instead:
- Add an absolute `readFloats()` method to `RandomAccessInput`
- Refactor the latest vector search file format to use `RandomAccessInput` instead of `IndexInput` to read vectors from disk.

Contributor guide

Open the contributing guide

Research direction

Start by locating RandomAccessInput and the latest vector search file-format implementation, then inspect how vectors are currently read through IndexInput.seek() and readFloats(). Done means RandomAccessInput exposes an absolute readFloats() method and the latest format reads vectors through it, with relevant tests passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.