matrixorigin / matrixorigin/matrixone
[Bug]: IVFFLAT PRE and AUTO retry reject CHAR primary keys as an invalid membership key type
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## 问题
IVFFLAT `mode=pre` 无法用于 `CHAR` 主键。相同表和查询使用 `mode=force`、`mode=post` 或 `mode=auto` 均返回正确结果,但显式 `mode=pre` 在执行阶段报错:
```text
invalid state required vector membership has invalid cardinality or key type
```
## 复现环境
- MatrixOne commit: `01d60e1c4ded1b0f3fc1a4ecd75ce54e95e23b90`
- 本地 2 CN / 1 TN / 1 LogService
- 两个 CN 均可稳定复现
## 最小复现
```sql
SET experimental_ivf_index = 1;
SET probe_limit = 1;
CREATE DATABASE ivf_char_pk_repro;
USE ivf_char_pk_repro;
CREATE TABLE t (
id CHAR(8) PRIMARY KEY,
grp INT,
v VECF32(2)
);
INSERT INTO t VALUES
('c1',1,'[0,0]'),
('c2',1,'[1,0]'),
('c3',2,'[0,1]'),
('c4',2,'[2,0]');
CREATE INDEX ix USING IVFFLAT ON t(v)
LISTS=1 OP_TYPE 'vector_l2_ops';
SELECT id FROM t WHERE grp=1
ORDER BY l2_distance(v,'[0,0]')
LIMIT 10 BY rank WITH OPTION 'mode=pre';
```
## 实际结果
```text
ERROR 20400: invalid state required vector membership has invalid cardinality or key type
```
`mode=include` 在索引没有 INCLUDE 列、因而回退到 PRE 时也报同一错误。
## 预期结果
```text
c1
c2
```
这与同一查询的 `mode=force`、`mode=post`、`mode=auto` 结果一致。
## 影响范围与定位
- 普通 IVFFLAT 和 `INCLUDE(grp)` IVFFLAT 的显式 PRE 都会报错;
- 带 `INCLUDE(grp)` 的 `mode=include` 正常,因为该路径直接从覆盖索引处理谓词;
- `VARCHAR`、`BINARY`、`UUID`、`DECIMAL` 和复合主键在相同 PRE 场景中正常;问题目前限定在 `CHAR` 主键;
- IVFFLAT 隐藏 entries 表的 `__mo_index_pri_col` 定义仍为 `CHAR(8)`,索引元数据没有丢失主键类型;
- 错误来自 `pkg/sql/compile/scope.go::validateRequiredVectorMembership`:PRE producer 产生的 runtime key vector 与 probe expression 的 `CHAR` 类型没有通过严格 OID 一致性检查。
因此这是 PRE runtime-membership 在 `CHAR` 主键上的类型传播/规范化问题,不是索引构建失败或 ANN 结果精度问题。
Contributor guide
Research direction
Start with the minimal SQL reproduction and inspect pkg/sql/compile/scope.go, especially validateRequiredVectorMembership, where the issue reports the PRE runtime key and CHAR probe expression failing strict OID validation. Trace the PRE runtime-membership path for CHAR primary keys and compare it with the working VARCHAR, BINARY, UUID, DECIMAL, and force/post paths. Done means PRE and include fallback return c1 and c2 without the invalid membership key error, while existing IVFFLAT behavior remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100