apache / apache/lucene

Allow `int4` quantization for vectors that are encoded as `byte` already

Open
#13,518 0 comments 0 reactions 0 assignees View on GitHub
type:enhancement vector-based-search
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

`int4` quantization allows for a further compression on top of `byte` vectors. If users have already completed a naive int8 quantization, or their model already provides optimized `byte[]` vectors, Lucene should be able to further quantize those to `int4` and compress their representation.

This change may cause some weird churn around validations that assume that scalar quantized formats only ever received float vectors, but it does seem like its something that Lucene should support.

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Begin by locating Lucene's scalar quantization path and the validations that assume scalar quantization receives float vectors. Done means allowing byte vectors to be further quantized to int4 and compressed without invalid validation failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.