Can we remove `compress` option for quantized KNN vector indexing?
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
Spinoff from [this comment](https://github.com/apache/lucene/pull/13767#issuecomment-2344025785).
This (`compress=true`) is a useful option when quantizing KNN vectors to 4 bits: it packs pairs of dimensions into a single byte, so the "hot working set" of your KNN/HNSW vectors at search time is half the already reduced (from `float32` -> `byte`) size. When `compress` is `false` then it's wasteful, using only four bits for every byte.
But it comes with some penalty to decode the "packed" (`compress=true`) form during KNN search, which is why we give this choice to the user.
But then I think there was at least one opto to that path, so maybe the performance penalty isn't so bad now? In which case maybe we can just always hardwire `compress=true` when quantized `bits=4`?
(`compress=true` doesn't apply to 7 bit quantization)
Contributor guide
Research direction
Start with the discussion in PR 13767, especially the linked issue comment, and trace the quantized KNN vector indexing and search entry points it identifies. Establish whether the performance cost of decoding compressed 4-bit vectors is acceptable, then document a decided scope for removing or retaining the compress option and the tests needed to verify the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100