matrixorigin / matrixorigin/matrixone

[Bug]: IVFFLAT drops ENUM member metadata for hidden PK and INCLUDE columns

Open
#28,994 2 comments 0 reactions 1 assignee Claimed by @iamlinjunhong View on GitHub
kind/bug needs-triage
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Description

`CREATE INDEX ... USING IVFFLAT ... INCLUDE(enum_column)` is accepted by DDL validation but fails while populating the hidden entries table. The IVFFLAT plugin advertises `ENUM` as a supported INCLUDE type, so this prevents a documented/supported index shape from being created.

## Environment

- Branch: `main`
- Commit: `01d60e1c4ded1b0f3fc1a4ecd75ce54e95e23b90`
- Deployment: local `mo-service`, 2 CN / 1 TN / 1 Log service
- Date: 2026-09-16

## Steps to reproduce

```sql
create database ivf_enum_repro;
use ivf_enum_repro;

create table t (
id bigint primary key,
e enum('cold','hot'),
v vecf32(2)
);
insert into t values
(1,'cold','[1,0]'),
(2,'hot','[2,0]');

create index ix using ivfflat on t(v)
op_type 'vector_l2_ops' lists=1 include(e);
```

## Actual behavior

The index build fails:

```text
ERROR 20203 (HY000): invalid argument operator cast, bad value [VARCHAR ENUM]
```

No `ix` metadata row or hidden index table remains after the rejected DDL, and the two base-table rows remain unchanged.

## Expected behavior

The index should be created and preserve the ENUM member metadata so INCLUDE filtering and projection use the same labels and ordinals as the source column. If ENUM is intentionally unsupported, DDL validation must reject it before creating/building hidden objects rather than advertising it as supported and failing inside the build.

## Stability and controls

- Reproducer: 3/3 on CN1 and 3/3 on CN2 with the same `VARCHAR ENUM` cast error.
- Control: an IVFFLAT index on the same vector column with `INCLUDE(json_col, uuid_col, binary_col, varbinary_col, text_col)` builds successfully and returns the expected indexed rows.
- SET control: `INCLUDE(set_col)` builds and matches the full-scan filter/projection result.
- Failure atomicity: base row count stays at 2; matching `mo_indexes` rows and hidden tables both remain 0.

## Evidence

Minimal SQL and the extended type matrix are retained in `evidence/ivfflat_include_exotic.sql` in the local validation worktree.

## Code analysis

`pkg/vectorindex/ivfflat/plugin/runtime/runtime.go` includes `types.T_enum` in `SupportedIncludeColumnTypes()`. During hidden entries-table construction, `pkg/vectorindex/ivfflat/plugin/plan/schema.go` copies only `Id`, `Width`, and `Scale` from each INCLUDE column and drops `plan.Type.Enumvalues`. The hidden ENUM column therefore loses its member list, and the synchronous population query in `pkg/vectorindex/ivfflat/plugin/compile/compile.go` fails when assigning the source ENUM value to that incomplete target type.

## Regression coverage

Add an IVFFLAT INCLUDE lifecycle case for ENUM covering initial build, label and ordinal predicates, projection, INSERT/UPDATE/DELETE maintenance, rollback, `ALTER ... REINDEX`, and multi-CN reads. Assert that failed DDL leaves no catalog or hidden-table residue.

## Related

- Source exploration: #28943
- #26873 concerned binary prepared reads through regular covering indexes; it did not cover IVFFLAT INCLUDE metadata or index creation.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.