Allow `int4` quantization for vectors that are encoded as `byte` already
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
`int4` quantization allows for a further compression on top of `byte` vectors. If users have already completed a naive int8 quantization, or their model already provides optimized `byte[]` vectors, Lucene should be able to further quantize those to `int4` and compress their representation.
This change may cause some weird churn around validations that assume that scalar quantized formats only ever received float vectors, but it does seem like its something that Lucene should support.
Contributor guide
Research direction
No files, tests, or entry points are named. Begin by locating Lucene's scalar quantization path and the validations that assume scalar quantization receives float vectors. Done means allowing byte vectors to be further quantized to int4 and compressed without invalid validation failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100