pingcap / pingcap/tidb

BuildTableInfoWithStmt misclassifies raw FULLTEXT constraints without planner preprocessing

Open
#70,570 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

### 1. Minimal reproduce step

Parse a `CREATE TABLE` containing a FULLTEXT index and pass the parser AST directly to the exported `ddl.BuildTableInfoWithStmt` API:

```go
sql := "CREATE TABLE `test`.`t` (" +
"`id` varchar(14) NOT NULL," +
"`text_col` text," +
"PRIMARY KEY (`id`)," +
"FULLTEXT INDEX `fulltext_idx` (`text_col`) WITH PARSER STANDARD)"

p := parser.New()
node, err := p.ParseOneStmt(sql, "", "")
// check err
stmt := node.(*ast.CreateTableStmt)

ctx := ddl.NewMetaBuildContextWithSctx(
mock.NewContext(),
metabuild.WithSuppressTooLongIndexErr(true),
)
_, err = ddl.BuildTableInfoWithStmt(ctx, stmt, "utf8mb4", "utf8mb4_bin", nil)
```

This reproduces on TiDB commit `3e4ee9fc99e7`.

### 2. What did you expect to see?

`BuildTableInfoWithStmt` should either:

1. normalize raw `ast.ConstraintFulltext` into the internal columnar FULLTEXT representation and build the corresponding `TableInfo`; or
2. provide/reuse an explicit lightweight normalization API that direct schema-builder callers can invoke without running the complete planner preprocessor.

The exported builder and schema-tracker paths should not treat FULLTEXT as an ordinary KV index on `TEXT`.

### 3. What did you see instead?

The parser leaves the constraint as raw `ast.ConstraintFulltext`. The planner preprocessor normally rewrites it to `ast.ConstraintColumnar` with `ast.IndexTypeFulltext`, but direct builder callers do not run planner preprocessing.

`BuildTableInfoWithStmt` consequently handles the raw constraint as an ordinary index and returns:

```text
[ddl:1170] BLOB/TEXT column 'text_col' used in key specification without a key length
```

This affects TiDB's lightweight schema-builder consumers. In TiFlow DM there are two affected paths:

- source schema tracker bootstrap;
- downstream `TableInfo` cache initialization from `SHOW CREATE TABLE`.

See pingcap/tiflow#12824 for the end-user failure (DM errors 44003 and 44018).

### 4. Root cause and suggested complete fix

FULLTEXT support added a planner preprocessing contract: raw `ConstraintFulltext` is rewritten before metadata construction. `BuildTableInfoWithStmt` remains callable with parser output, and TiDB's DM schema tracker also invokes it directly, so this contract is currently easy to violate.

A complete fix should share FULLTEXT normalization between planner preprocessing and direct metadata-building paths, preserving the same grammar/index-option validation. It should cover the FULLTEXT DDL lifecycle rather than merely dropping metadata, including CREATE/ALTER/DROP behavior where applicable.

A narrow downstream mitigation in TiFlow can ignore raw FULLTEXT constraints because they are non-unique and not used as DML row handles, but that is intentionally not the complete semantic solution.

Related historical changes:

- pingcap/tidb#3658
- pingcap/tidb#9821
- pingcap/tidb#60720
- pingcap/tiflow#12824

### 5. What is your TiDB version?

```text
3e4ee9fc99e7
```

Contributor guide

Open the contributing guide

Research direction

Start at the exported ddl.BuildTableInfoWithStmt entry point and compare its handling of parser.New output with the planner preprocessing that rewrites ast.ConstraintFulltext. Trace the CREATE/ALTER/DROP FULLTEXT paths and the affected schema-builder consumers; done means direct parser ASTs preserve FULLTEXT validation and metadata semantics without the TEXT key-length error.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.