matrixorigin / matrixorigin/matrixone
[Bug]: HNSW vector_ip_ops returns incorrect INNER_PRODUCT score
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Summary
An HNSW index created with `OP_TYPE 'vector_ip_ops'` returns the correct nearest row but exposes an incorrect `INNER_PRODUCT` value. The HNSW plan returns a score that is exactly 1 greater than the scalar/full-scan result in the minimal case below.
The issue affects both `VECF32` and `VECF64`.
## Environment
- Branch: `main`
- Commit: `3046f14d94b3f1a25839eeeb65729b64f85ac647`
- Local standalone CN/TN/logservice cluster
## Reproduction
```sql
SET experimental_hnsw_index = 1;
CREATE TABLE t (
id BIGINT PRIMARY KEY,
v VECF64(3)
);
INSERT INTO t VALUES
(1, '[1,1,1]'),
(2, '[2,2,2]'),
(3, '[-3,-3,-3]'),
(4, '[4,4,4]');
CREATE INDEX idx_v USING HNSW ON t(v)
M=4 EF_CONSTRUCTION=32 EF_SEARCH=16
OP_TYPE 'vector_ip_ops' MAX_INDEX_CAPACITY 1000000;
-- Scalar value.
SELECT id, INNER_PRODUCT(v, '[1,1,1]') AS d
FROM t
WHERE id = 4;
-- Secondary key prevents the HNSW rewrite.
SELECT id, INNER_PRODUCT(v, '[1,1,1]') AS d
FROM t
ORDER BY d, id
LIMIT 1;
-- Uses hnsw_search.
SELECT id, INNER_PRODUCT(v, '[1,1,1]') AS d
FROM t
ORDER BY d
LIMIT 1;
```
## Actual result
```text
Scalar WHERE id=4: 4 -12
Full table scan: 4 -12
HNSW index plan: 4 -11
```
The same difference reproduced 3/3 times with `VECF32(3)` and 3/3 times with `VECF64(3)`.
## Expected result
The HNSW rewrite must preserve the SQL value of `INNER_PRODUCT`. The indexed query should return `4, -12`, matching direct scalar evaluation and the full table scan.
## White-box evidence
- The indexed plan replaces the ORDER BY expression with the score emitted by `hnsw_search`.
- `pkg/vectorindex/hnsw/search.go` calls `metric.DistanceTransformHnsw` before returning that score.
- `DistanceTransformHnsw` currently transforms only the L2sq-to-L2 case and otherwise returns the underlying USearch distance unchanged.
This is consistent with the observed fixed offset, but the issue is reported based on the SQL-level result mismatch rather than assuming a specific implementation fix.
Contributor guide
Assessment
This issue has not been assessed yet.