apache / apache/lucene

Can we remove `compress` option for quantized KNN vector indexing?

Open
#13,768 8 comments 0 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Spinoff from [this comment](https://github.com/apache/lucene/pull/13767#issuecomment-2344025785).

This (`compress=true`) is a useful option when quantizing KNN vectors to 4 bits: it packs pairs of dimensions into a single byte, so the "hot working set" of your KNN/HNSW vectors at search time is half the already reduced (from `float32` -> `byte`) size. When `compress` is `false` then it's wasteful, using only four bits for every byte.

But it comes with some penalty to decode the "packed" (`compress=true`) form during KNN search, which is why we give this choice to the user.

But then I think there was at least one opto to that path, so maybe the performance penalty isn't so bad now? In which case maybe we can just always hardwire `compress=true` when quantized `bits=4`?

(`compress=true` doesn't apply to 7 bit quantization)

Contributor guide

Open the contributing guide

Research direction

Start with the discussion in PR 13767, especially the linked issue comment, and trace the quantized KNN vector indexing and search entry points it identifies. Establish whether the performance cost of decoding compressed 4-bit vectors is acceptable, then document a decided scope for removing or retaining the compress option and the tests needed to verify the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.