matrixorigin / matrixorigin/matrixone
[Bug]: IVFFLAT PRE rejects BIT primary keys and AUTO retry fails with 20631
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## 问题
IVFFLAT 的 PRE membership 无法处理 `BIT` 主键。显式 `mode=pre` 在正常源表上报 `IN [BIT BIT]` 不支持;当低选择性查询使默认 `mode=auto` 从 POST 重试 PRE 时,查询转而报事务定义变化错误,无法返回本应存在的结果。
## 环境
- MatrixOne commit: `01d60e1c4ded1b0f3fc1a4ecd75ce54e95e23b90`
- 本地 2 CN / 1 TN / 1 LogService
- 两个 CN 结果一致
## 最小复现
```sql
SET experimental_ivf_index = 1;
SET probe_limit = 1;
CREATE DATABASE ivf_bit_pk_repro;
USE ivf_bit_pk_repro;
CREATE TABLE t(
id BIT(8) PRIMARY KEY,
selected INT,
v VECF32(2)
);
INSERT INTO t VALUES
(CAST(1 AS UNSIGNED),1,'[0.1,0]'),
(CAST(2 AS UNSIGNED),1,'[0.2,0]'),
(CAST(3 AS UNSIGNED),0,'[0.3,0]'),
(CAST(4 AS UNSIGNED),0,'[0.4,0]');
CREATE INDEX ix USING IVFFLAT ON t(v)
LISTS=1 OP_TYPE 'vector_l2_ops';
SELECT HEX(id) FROM t WHERE selected=1
ORDER BY l2_distance(v,'[0,0]')
LIMIT 10 BY rank WITH OPTION 'mode=pre';
```
## 实际结果
```text
ERROR 20203: invalid argument operator in, bad value [BIT BIT]
```
同一表:
- `mode=force` 与 `mode=post` 都返回两行;
- `mode=auto` 在 POST 已经能够补满的简单查询中返回正确结果;
- 无 INCLUDE 列的 `mode=include` 回退 PRE 后报同一 `BIT BIT` 错误;
- `BIT(1)`、`BIT(8)`、`BIT(64)` 都有相同结果。
## AUTO correctness retry
在 200 行 `BIT(16) PRIMARY KEY` 表中,让只有距离较远的 5 行满足谓词:
```sql
SELECT HEX(id) FROM t WHERE selected=1
ORDER BY l2_distance(v,'[0,0]')
LIMIT 10 BY rank WITH OPTION 'mode=auto';
```
- FORCE 返回 5 行;
- POST 返回空,需要 AUTO 重试 PRE;
- AUTO 在两个 CN 上连续 3/3 报:
```text
ERROR 20631: txn need retry in rc mode, def changed
```
## 预期
BIT 是合法主键类型,IVFFLAT 已能够建索引,FORCE/POST 也能回表得到正确结果。PRE 应使用 BIT 值构造有效的 membership 条件;AUTO correctness retry 不应把可返回 5 行的查询转换为事务重试错误。
## 定位
PRE 将候选主键 runtime vector 转换为隐藏 entries 主键上的 membership 表达式。当前该路径构造 `IN(BIT, BIT-vector)`,函数绑定阶段没有可用的 BIT IN 实现,因此在 `ivfRuntimeMembershipExpr` 路径报错。AUTO 的二次编译/执行又把这一失败表现为 20631。
这与 #29045 的 CHAR 主键问题不同:CHAR 在 `validateRequiredVectorMembership` 的 OID 一致性校验中失败;BIT 能通过 payload 传输,但在生成/绑定 IN membership 表达式时失败。
Contributor guide
Research direction
Start with the ivfRuntimeMembershipExpr path described in the issue and reproduce the SQL example with a BIT primary key. Trace how PRE builds and binds the membership expression, then compare the AUTO retry path that produces error 20631. Done means PRE returns the expected rows and AUTO can retry without converting the query into a transaction-definition error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100