matrixorigin / matrixorigin/matrixone
[Feature Request]: IVFFLAT/IVFPQ/CAGRA support quantization bf16/float16/int8/uint8
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
### Is there an existing issue for the same feature request?
- [x] I have checked the existing issues.
### Is your feature request related to a problem?
```Markdown
```
### Describe the feature you'd like
IVFFLAT support quantization bf16/float16/int8/uint8
IVFPQ/CAGRA support base type float16/
### Describe implementation you've considered
_No response_
### Documentation, Adoption, Use Case, Migration Strategy
# Feature: Vector index narrow base-types & quantization (ivfflat / ivfpq / cagra) + basic array functions
This change adds **narrow vector base types** (`vecf16`, `vecbf16`, `vecint8`, `vecuint8`)
and a **`QUANTIZATION` option** to the three vector index algorithms, plus extends the
**basic array/vector SQL functions** to work on the new narrow types.
The two parts can be tested independently:
1. [Part 1 — Vector index base types & quantization](#part-1--vector-index-base-types--quantization)
2. [Part 2 — Basic array functions](#part-2--basic-array-functions)
A copy-paste [end-to-end smoke test](#end-to-end-smoke-test) is at the bottom.
---
## Background concepts
- **Base type** — the declared type of the vector column (`VECF32(d)`, `VECF16(d)`, …).
This is what `INSERT`/`SELECT` see.
- **QUANTIZATION** — an *index-only* storage option. The index keeps a **downcast copy**
of each vector (the "entries") in the quantization type to save space / speed up search;
the base column is unchanged. Omitting `QUANTIZATION` keeps the entries in the base type.
- **op_type** — the distance metric: `vector_l2_ops` (L2, default), `vector_l2sq_ops`
(L2 squared), `vector_ip_ops` (inner product), `vector_cosine_ops` (cosine).
---
## Part 1 — Vector index base types & quantization
### Supported matrix
| Capability | IVFFLAT | IVFPQ | CAGRA |
|---|---|---|---|
| Backend | CPU | GPU/cuVS | GPU/cuVS |
| Base types | f32, f64, f16, bf16, int8, uint8 | f32, f16 | f32, f16 |
| QUANTIZATION | f32, f16, bf16, int8, uint8 | f16, int8, uint8 | f16, int8, uint8 |
| op_type l2 / l2sq | yes | yes | yes |
| op_type ip / cosine | yes | yes | yes |
| op_type l1 | yes | no | no |
| int8/uint8 quant metric | any | L2 only | L2 only |
| Primary key | any | bigint | bigint |
| Experimental flag | none | experimental_ivfpq_index | experimental_cagra_index |
> Why `int8`/`uint8` are L2-only on GPU: the affine scalar quantizer `q(x)=scale*x+offset`
> preserves L2 geometry (offset cancels in differences) but biases inner-product and
> rotates cosine angles, so IP/cosine would return wrong neighbors.
### Enabling the GPU indexes
```sql
SET experimental_ivfpq_index = 1; -- required before CREATE INDEX ... USING IVFPQ
SET experimental_cagra_index = 1; -- required before CREATE INDEX ... USING CAGRA
```
### gpu_mode — GPU vs CPU dispatch toggle
`gpu_mode` is a boolean session variable that controls whether vector search /
distance work is dispatched to the GPU or computed on the CPU. It affects the
IVFFLAT search brute-force (centroid assignment), the pairwise / array distance
functions (Part 2), and the cuVS overflow path.
- **Default:** `1` (true) on a GPU build (`-tags gpu`); `0` (false) on a CPU-only build.
- **Override per session:** `SET gpu_mode = 0;` forces CPU paths, `SET gpu_mode = 1;` forces GPU.
- GPU and CPU paths return **identical results** — this is the main thing to test.
```sql
SET gpu_mode = 1; -- use GPU
SET gpu_mode = 0; -- use CPU (identical output)
```
### CREATE TABLE — base type syntax
```sql
-- dim is the fixed vector dimension
CREATE TABLE t (id BIGINT PRIMARY KEY, v VECF32(8)); -- 32-bit float
CREATE TABLE t (id BIGINT PRIMARY KEY, v VECF16(8)); -- 16-bit float (half)
CREATE TABLE t (id INT PRIMARY KEY, v VECF64(8)); -- 64-bit float (IVFFLAT only)
CREATE TABLE t (id INT PRIMARY KEY, v VECBF16(8)); -- bfloat16 (IVFFLAT only)
CREATE TABLE t (id INT PRIMARY KEY, v VECINT8(8)); -- signed 8-bit (IVFFLAT only)
CREATE TABLE t (id INT PRIMARY KEY, v VECUINT8(8)); -- unsigned 8-bit (IVFFLAT only)
```
### CREATE INDEX — per algorithm
**IVFFLAT** (no experimental flag, all base types):
```sql
CREATE INDEX ix USING IVFFLAT ON t (v)
lists = 2 -- cluster count
op_type 'vector_l2_ops'
kmeans_train_percent = 10 -- optional, default 10 (k-means sample %)
kmeans_max_iteration = 20 -- optional, default 20
[ QUANTIZATION 'float16' ]; -- optional: downcast entries
```
**IVFPQ** (`experimental_ivfpq_index=1`, base VECF32/VECF16):
```sql
CREATE INDEX ix USING IVFPQ ON t (v)
lists = 2
op_type 'vector_l2_ops'
m = 2 -- # of subquantizers (optional)
bits_per_code = 8 -- bits per code (optional)
kmeans_train_percent = 10 -- optional, default 10
kmeans_max_iteration = 20 -- optional, default 20
max_index_capacity = 0 -- optional, default 0 = auto-detect
[ QUANTIZATION 'int8' ]; -- optional: float16 / int8 / uint8
```
**CAGRA** (`experimental_cagra_index=1`, base VECF32/VECF16):
```sql
CREATE INDEX ix USING CAGRA ON t (v)
op_type 'vector_l2_ops'
intermediate_graph_degree = 8 -- optional
graph_degree = 4 -- optional
itopk_size = 16 -- optional
max_index_capacity = 0 -- optional, default 0 = auto-detect
[ QUANTIZATION 'uint8' ]; -- optional: float16 / int8 / uint8
```
**Index-param availability** (all are optional `key = int` clauses in `CREATE INDEX`):
| Param | IVFFLAT | IVFPQ | CAGRA | Default |
|---|---|---|---|---|
| `lists` | yes | yes | no | 1 / required >0 |
| `m`, `bits_per_code` | no | yes | no | auto |
| `intermediate_graph_degree` | no | no | yes | cuVS default |
| `graph_degree` | no | no | yes | cuVS default |
| `itopk_size` | no | no | yes | cuVS default |
| `kmeans_train_percent` | yes | yes | no | 10 |
| `kmeans_max_iteration` | yes | yes | no | 20 |
| `max_index_capacity` | no | yes | yes | 0 (auto) |
Optional clauses available on all three: `ASYNC` (build in background via CDC),
`AUTO_UPDATE` + `DAY <0-6>` `HOUR <0-23>` (scheduled rebuild). IVFPQ/CAGRA also support
`INCLUDE (col, …)` to carry int/float scalar columns for pre-filtering.
### Test scenarios — Part 1
#### 1A. IVFFLAT narrow base type, no quantization (entries keep base type)
```sql
DROP TABLE IF EXISTS t_if;
CREATE TABLE t_if (id INT PRIMARY KEY, v VECINT8(4));
INSERT INTO t_if VALUES (1,'[1,2,3,4]'), (2,'[5,6,7,8]'), (3,'[2,2,2,2]');
CREATE INDEX ix USING IVFFLAT ON t_if (v) lists=2 op_type 'vector_l2_ops';
SELECT id FROM t_if ORDER BY l2_distance(v, '[1,2,3,4]') LIMIT 2; -- expect 1 first
```
#### 1B. IVFFLAT f32 base + quantization downcast
```sql
DROP TABLE IF EXISTS t_q;
CREATE TABLE t_q (id INT PRIMARY KEY, v VECF32(4));
INSERT INTO t_q VALUES (1,'[1,2,3,4]'), (2,'[5,6,7,8]'), (3,'[2,2,2,2]');
CREATE INDEX ix USING IVFFLAT ON t_q (v) lists=2 op_type 'vector_l2_ops' QUANTIZATION 'int8';
SELECT id FROM t_q ORDER BY l2_distance(v, '[1,2,3,4]') LIMIT 2;
```
#### 1C. IVFPQ on VECF16 base + uint8 quantization (GPU)
```sql
SET experimental_ivfpq_index = 1;
DROP TABLE IF EXISTS t_pq;
CREATE TABLE t_pq (id BIGINT PRIMARY KEY, v VECF16(4));
INSERT INTO t_pq VALUES (1,'[1,2,3,4]'), (2,'[5,6,7,8]'), (3,'[2,2,2,2]');
CREATE INDEX ix USING IVFPQ ON t_pq (v) lists=2 m=2 bits_per_code=8
op_type 'vector_l2_ops' QUANTIZATION 'uint8';
SELECT id FROM t_pq ORDER BY l2_distance(v, '[1,2,3,4]') LIMIT 2;
```
#### 1D. CAGRA on VECF32 base + int8 quantization (GPU)
```sql
SET experimental_cagra_index = 1;
DROP TABLE IF EXISTS t_cg;
CREATE TABLE t_cg (id BIGINT PRIMARY KEY, v VECF32(4));
INSERT INTO t_cg VALUES (1,'[1,2,3,4]'), (2,'[5,6,7,8]'), (3,'[2,2,2,2]');
CREATE INDEX ix USING CAGRA ON t_cg (v)
op_type 'vector_l2_ops' graph_degree=4 intermediate_graph_degree=8 QUANTIZATION 'int8';
SELECT id FROM t_cg ORDER BY l2_distance(v, '[1,2,3,4]') LIMIT 2;
```
#### 1E. Negative cases — must be REJECTED with an error
```sql
SET experimental_cagra_index = 1;
-- bf16 quantization not allowed on GPU indexes
CREATE INDEX ng1 USING CAGRA ON t_cg (v) op_type 'vector_l2_ops' QUANTIZATION 'bf16';
-- int8/uint8 quantization with non-L2 op_type
CREATE INDEX ng2 USING CAGRA ON t_cg (v) op_type 'vector_ip_ops' QUANTIZATION 'int8';
CREATE INDEX ng3 USING CAGRA ON t_cg (v) op_type 'vector_cosine_ops' QUANTIZATION 'uint8';
-- VECF64 base not allowed on GPU indexes (CAGRA/IVFPQ are vecf32/vecf16 only)
DROP TABLE IF EXISTS t_bad; CREATE TABLE t_bad (id BIGINT PRIMARY KEY, v VECF64(4));
CREATE INDEX ng4 USING CAGRA ON t_bad (v) op_type 'vector_l2_ops';
-- non-bigint PK not allowed on GPU indexes
DROP TABLE IF EXISTS t_pk; CREATE TABLE t_pk (id INT PRIMARY KEY, v VECF32(4));
CREATE INDEX ng5 USING CAGRA ON t_pk (v) op_type 'vector_l2_ops';
```
Expected: each `ng*` fails (e.g. "vector column must be vecf32 or vecf16",
"int8/uint8 ... L2 ... only", "primary key must be bigint").
---
## Part 2 — Basic array functions
These built-in functions operate on vector/array columns. The narrow types
(`vecbf16`, `vecf16`, `vecint8`, `vecuint8`) are now accepted in addition to
`vecf32` / `vecf64`; narrow inputs are computed via float32 and return `float64`.
### Function reference
| Function | Form | Returns | Notes |
|---|---|---|---|
| `vector_dims(v)` | unary | `int64` | element count (dimension) |
| `summation(v)` | unary | `float64` | sum of elements |
| `l1_norm(v)` | unary | `float64` | sum of abs values |
| `l2_norm(v)` | unary | `float64` | Euclidean norm |
| `normalize_l2(v)` | unary | array (same type) | unit-length vector |
| `subvector(v, start[, len])` | unary | array (same type) | 1-indexed slice; negative start = from end |
| `inner_product(a, b)` | binary | `float64` | dot product |
| `cosine_similarity(a, b)` | binary | `float64` | in [-1, 1] |
| `cosine_distance(a, b)` | binary | `float64` | `1 - cosine_similarity` |
| `l2_distance(a, b)` | binary | `float64` | Euclidean distance |
| `l2_distance_sq(a, b)` | binary | `float64` | squared L2 (no sqrt) |
Encoding helpers (varchar → vector): `vecf32_from_base64`, `vecf64_from_base64`,
`vecbf16_from_base64`, `vecf16_from_base64`, `vecint8_from_base64`, `vecuint8_from_base64`.
### Test scenarios — Part 2
```sql
DROP TABLE IF EXISTS af;
CREATE TABLE af (id INT PRIMARY KEY, a VECF32(3), b VECF32(3));
INSERT INTO af VALUES (1, '[1,2,3]', '[1,1,1]'), (2, '[4,5,6]', '[0,0,1]');
-- unary
SELECT id, vector_dims(a), summation(a), l1_norm(a), l2_norm(a) FROM af ORDER BY id;
SELECT id, normalize_l2(a) FROM af ORDER BY id;
SELECT id, subvector(a, 2), subvector(a, 1, 2), subvector(a, -1) FROM af ORDER BY id;
-- binary
SELECT id, inner_product(a, b), l2_distance(a, b), l2_distance_sq(a, b),
cosine_similarity(a, b), cosine_distance(a, b)
FROM af ORDER BY id;
-- with a literal vector
SELECT id, l2_distance(a, '[1,2,3]') FROM af ORDER BY id;
-- in WHERE / GROUP BY
SELECT * FROM af WHERE l2_norm(a) > 3;
SELECT vector_dims(a), count(*) FROM af GROUP BY vector_dims(a);
```
### Narrow-type coverage (functions accept the new types)
```sql
DROP TABLE IF EXISTS afn;
CREATE TABLE afn (id INT PRIMARY KEY, f16 VECF16(3), i8 VECINT8(3), u8 VECUINT8(3));
INSERT INTO afn VALUES (1, '[1,2,3]', '[1,2,3]', '[1,2,3]');
SELECT vector_dims(f16), l2_norm(f16), inner_product(f16, '[1,1,1]') FROM afn;
SELECT vector_dims(i8), l2_distance(i8, '[0,0,0]') FROM afn;
SELECT vector_dims(u8), cosine_similarity(u8, '[1,1,1]') FROM afn;
```
---
## End-to-end smoke test
Run top-to-bottom; every `SELECT` should return rows and every `ng*` `CREATE INDEX`
in Part 1E should fail.
```sql
-- enable GPU indexes
SET experimental_ivfpq_index = 1;
SET experimental_cagra_index = 1;
-- array functions
DROP TABLE IF EXISTS smoke_af;
CREATE TABLE smoke_af (id INT PRIMARY KEY, a VECF32(4));
INSERT INTO smoke_af VALUES (1,'[1,2,3,4]'),(2,'[5,6,7,8]'),(3,'[2,2,2,2]');
SELECT id, vector_dims(a), l2_norm(a), l2_distance(a,'[1,2,3,4]') FROM smoke_af ORDER BY id;
-- ivfflat narrow base
DROP TABLE IF EXISTS smoke_if;
CREATE TABLE smoke_if (id INT PRIMARY KEY, v VECINT8(4));
INSERT INTO smoke_if VALUES (1,'[1,2,3,4]'),(2,'[5,6,7,8]'),(3,'[2,2,2,2]');
CREATE INDEX ix USING IVFFLAT ON smoke_if (v) lists=2 op_type 'vector_l2_ops';
SELECT id FROM smoke_if ORDER BY l2_distance(v,'[1,2,3,4]') LIMIT 2;
-- ivfpq f16 base + uint8 quant
DROP TABLE IF EXISTS smoke_pq;
CREATE TABLE smoke_pq (id BIGINT PRIMARY KEY, v VECF16(4));
INSERT INTO smoke_pq VALUES (1,'[1,2,3,4]'),(2,'[5,6,7,8]'),(3,'[2,2,2,2]');
CREATE INDEX ix USING IVFPQ ON smoke_pq (v) lists=2 m=2 bits_per_code=8
op_type 'vector_l2_ops' QUANTIZATION 'uint8';
SELECT id FROM smoke_pq ORDER BY l2_distance(v,'[1,2,3,4]') LIMIT 2;
-- cagra f32 base + int8 quant
DROP TABLE IF EXISTS smoke_cg;
CREATE TABLE smoke_cg (id BIGINT PRIMARY KEY, v VECF32(4));
INSERT INTO smoke_cg VALUES (1,'[1,2,3,4]'),(2,'[5,6,7,8]'),(3,'[2,2,2,2]');
CREATE INDEX ix USING CAGRA ON smoke_cg (v) op_type 'vector_l2_ops'
graph_degree=4 intermediate_graph_degree=8 QUANTIZATION 'int8';
SELECT id FROM smoke_cg ORDER BY l2_distance(v,'[1,2,3,4]') LIMIT 2;
```
> **Note for the test team:** IVFPQ and CAGRA require a GPU-enabled build. On a CPU-only
> build, only the IVFFLAT and array-function cases apply; the `experimental_*` GPU index
> cases will not run.
---
## Build variants
Two independent axes:
- **GPU vs CPU** — `MO_CL_CUDA=1` builds the cuVS/GPU code (adds `-tags gpu`); omitting
it (or `MO_CL_CUDA=0`) builds CPU-only. GPU needs the CUDA toolkit (12.0/13.0+) and the
cuVS conda env **activated** (`CONDA_PREFIX` must be set).
- **archsimd vs noarchsimd** — `archsimd` enables the Go SIMD experiment kernels; it needs
a Go 1.26+ toolchain with `GOEXPERIMENT=simd` and `GOAMD64=v3` (x86_64 only). Omitting
them is the default `noarchsimd` build (scalar/unroll kernels). GPU default is noarchsimd.
Always `make clean` first when switching variants, and use `-j8` for the parallel cgo build.
```sh
# 1. CPU (noarchsimd) — the default
make clean && make -j8
# (equivalently: MO_CL_CUDA=0 make -j8)
# 2. CPU + archsimd
make clean && \
make GOEXPERIMENT_OPT="GOEXPERIMENT=simd" GOAMD64=v3 -j8
# 3. GPU (== GPU + noarchsimd) — activate the cuVS conda env first
conda activate go
make clean && MO_CL_CUDA=1 make -j8
# 4. GPU + archsimd
conda activate go
make clean && \
MO_CL_CUDA=1 make GOEXPERIMENT_OPT="GOEXPERIMENT=simd" GOAMD64=v3 -j8
```
| Variant | MO_CL_CUDA | GO / GOEXPERIMENT / GOAMD64 | GPU indexes (ivfpq/cagra) | SIMD kernels |
|---|---|---|---|---|
| CPU (noarchsimd) | 0 / unset | default `go` | no | no |
| CPU + archsimd | 0 / unset | go1.26 + `simd` + `v3` | no | yes |
| GPU (noarchsimd) | 1 | default `go` | yes | no |
| GPU + archsimd | 1 | go1.26 + `simd` + `v3` | yes | yes |
> `GPU` and `GPU + noarchsimd` are the same build (the default GPU build has no SIMD).
### Additional information
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.