Implement quantization on fp16 directly without inflating to fp32
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
Follow-up to #16473, which added scalar quantization support for the float16 vector encoding.
The quantization works on fp32, so every fp16 vector is inflated to a float[] before it is quantized, and the centroid and corrective terms are computed in fp32 as well. Since Java has no fp16 arithmetic type, so there is no way to compute in fp16 at the moment. Once Java have support for doing arithmetic operations on fp16 directly, we should switch our existing implementation.
Contributor guide
Research direction
Start by reading #16473 and the current scalar quantization implementation for float16 vector encoding. Confirm the available Java fp16 arithmetic support and identify where vectors are inflated to float[] and where centroid and corrective terms are computed. Done means quantization operates directly on fp16 vectors without fp32 inflation once the required Java support exists.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100