[Enhance]: Integrate TurboQuant/PolarQuant zero-overhead vector quantization for compressed ANNS indexing
- Dominant language
- C++
- Stars
- 15.9k
- Forks
- 998
- Avg merge
- 4d 7h
- Merged PRs (30d)
- 34
Description
### Affected Component
HNSW index, vector quantization layer, index builder, Python API
### Current Behavior
zvec currently relies on standard vector quantization approaches (e.g., scalar or product quantization) that carry unavoidable memory overhead. Traditional quantization methods must compute and store full-precision quantization constants (scale/zero-point) for every small block of data, adding 1–2 extra bits per value. This overhead partially defeats the compression goal: a 4-bit quantization scheme in practice consumes closer to 5–6 bits per dimension, increasing memory footprint and slowing down index building and similarity search. There is no built-in support for theoretically-optimal, data-oblivious quantizers that eliminate this overhead while preserving recall accuracy.
### Desired Improvement
Integrate support for **TurboQuant** and its constituent algorithms (**PolarQuant** and **QJL**) as a quantization backend option within zvec's index building and search pipeline.
**Proposed changes:**
1. **PolarQuant quantizer**: Add a quantization codec that converts Cartesian vector coordinates to polar form, applying recursive polar transforms so that the angular component can be quantized without storing per-block normalization constants. This eliminates the memory overhead associated with scale factors.
2. **QJL residual correction**: After primary quantization with PolarQuant, apply the Quantized Johnson-Lindenstrauss 1-bit residual pass to the remaining error. This step requires zero memory overhead and removes quantization bias, producing more accurate inner-product / dot-product estimates for ANNS scoring.
3. **TurboQuant codec**: Expose a unified `TurboQuant` quantization option that chains PolarQuant + QJL, configurable via a `bits` parameter (e.g., 3-bit, 4-bit).
4. **Python API**: Surface the new codec through the existing Python API, e.g.:
```python
index = zvec.Index(quantization="turboquant", bits=4)
```
5. **No training / fine-tuning requirement**: The implementation must be data-oblivious — no dataset-specific codebook training should be required.
**Reference**: TurboQuant (ICLR 2026) — https://arxiv.org/abs/2504.19874; PolarQuant (AISTATS 2026) — https://arxiv.org/abs/2502.02617; QJL — https://arxiv.org/abs/2406.03482
### Impact
- **Lower memory usage**: Quantizing vectors to 3–4 bits with zero overhead (no stored scale constants) reduces index memory by 6× or more compared to 32-bit storage, enabling larger datasets to fit in RAM.
- **Faster queries**: By operating at lower bit-widths, inner-product computations over compressed vectors are significantly cheaper. Benchmarks show up to 8× speedup in attention-logit throughput on H100 GPUs at 4-bit vs 32-bit.
- **No accuracy loss**: TurboQuant achieves near-lossless recall (matching full-precision results on standard ANNS benchmarks such as GloVe d=200) without any dataset-specific codebook training, making it immediately usable on arbitrary embedding spaces.
- **Faster index building**: The data-oblivious design eliminates the expensive training phase required by methods like PQ or RaBitQ, dramatically reducing time-to-index for large collections.
- **Drop-in adoption**: Exposing TurboQuant as a named codec in the Python API allows existing zvec users to opt in with a single parameter change, lowering the barrier for adoption in production RAG and semantic search pipelines.
Contributor guide
Assessment
This issue has not been assessed yet.