apache / apache/lucene

Implement quantization on fp16 directly without inflating to fp32

Open
#16,533 2 comments 2 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Follow-up to #16473, which added scalar quantization support for the float16 vector encoding.

The quantization works on fp32, so every fp16 vector is inflated to a float[] before it is quantized, and the centroid and corrective terms are computed in fp32 as well. Since Java has no fp16 arithmetic type, so there is no way to compute in fp16 at the moment. Once Java have support for doing arithmetic operations on fp16 directly, we should switch our existing implementation.

Contributor guide

Open the contributing guide

Research direction

Start by reading #16473 and the current scalar quantization implementation for float16 vector encoding. Confirm the available Java fp16 arithmetic support and identify where vectors are inflated to float[] and where centroid and corrective terms are computed. Done means quantization operates directly on fp16 vectors without fp32 inflation once the required Java support exists.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.