alibaba / alibaba/zvec

[Enhance]: Integrate TurboQuant/PolarQuant zero-overhead vector quantization for compressed ANNS indexing

Open
#274 2 comments 0 reactions 1 assignee Assigned to @richyreachy View on GitHub
deferred enhancement
Dominant language
C++
Stars
15.9k
Forks
998
Avg merge
4d 7h
Merged PRs (30d)
34

Description

### Affected Component

HNSW index, vector quantization layer, index builder, Python API

### Current Behavior

zvec currently relies on standard vector quantization approaches (e.g., scalar or product quantization) that carry unavoidable memory overhead. Traditional quantization methods must compute and store full-precision quantization constants (scale/zero-point) for every small block of data, adding 1–2 extra bits per value. This overhead partially defeats the compression goal: a 4-bit quantization scheme in practice consumes closer to 5–6 bits per dimension, increasing memory footprint and slowing down index building and similarity search. There is no built-in support for theoretically-optimal, data-oblivious quantizers that eliminate this overhead while preserving recall accuracy.

### Desired Improvement

Integrate support for **TurboQuant** and its constituent algorithms (**PolarQuant** and **QJL**) as a quantization backend option within zvec's index building and search pipeline.

**Proposed changes:**

1. **PolarQuant quantizer**: Add a quantization codec that converts Cartesian vector coordinates to polar form, applying recursive polar transforms so that the angular component can be quantized without storing per-block normalization constants. This eliminates the memory overhead associated with scale factors.

2. **QJL residual correction**: After primary quantization with PolarQuant, apply the Quantized Johnson-Lindenstrauss 1-bit residual pass to the remaining error. This step requires zero memory overhead and removes quantization bias, producing more accurate inner-product / dot-product estimates for ANNS scoring.

3. **TurboQuant codec**: Expose a unified `TurboQuant` quantization option that chains PolarQuant + QJL, configurable via a `bits` parameter (e.g., 3-bit, 4-bit).

4. **Python API**: Surface the new codec through the existing Python API, e.g.:
```python
index = zvec.Index(quantization="turboquant", bits=4)
```

5. **No training / fine-tuning requirement**: The implementation must be data-oblivious — no dataset-specific codebook training should be required.

**Reference**: TurboQuant (ICLR 2026) — https://arxiv.org/abs/2504.19874; PolarQuant (AISTATS 2026) — https://arxiv.org/abs/2502.02617; QJL — https://arxiv.org/abs/2406.03482

### Impact

- **Lower memory usage**: Quantizing vectors to 3–4 bits with zero overhead (no stored scale constants) reduces index memory by 6× or more compared to 32-bit storage, enabling larger datasets to fit in RAM.
- **Faster queries**: By operating at lower bit-widths, inner-product computations over compressed vectors are significantly cheaper. Benchmarks show up to 8× speedup in attention-logit throughput on H100 GPUs at 4-bit vs 32-bit.
- **No accuracy loss**: TurboQuant achieves near-lossless recall (matching full-precision results on standard ANNS benchmarks such as GloVe d=200) without any dataset-specific codebook training, making it immediately usable on arbitrary embedding spaces.
- **Faster index building**: The data-oblivious design eliminates the expensive training phase required by methods like PQ or RaBitQ, dramatically reducing time-to-index for large collections.
- **Drop-in adoption**: Exposing TurboQuant as a named codec in the Python API allows existing zvec users to opt in with a single parameter change, lowering the barrier for adoption in production RAG and semantic search pipelines.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.