BuildTableInfoWithStmt misclassifies raw FULLTEXT constraints without planner preprocessing
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
### 1. Minimal reproduce step
Parse a `CREATE TABLE` containing a FULLTEXT index and pass the parser AST directly to the exported `ddl.BuildTableInfoWithStmt` API:
```go
sql := "CREATE TABLE `test`.`t` (" +
"`id` varchar(14) NOT NULL," +
"`text_col` text," +
"PRIMARY KEY (`id`)," +
"FULLTEXT INDEX `fulltext_idx` (`text_col`) WITH PARSER STANDARD)"
p := parser.New()
node, err := p.ParseOneStmt(sql, "", "")
// check err
stmt := node.(*ast.CreateTableStmt)
ctx := ddl.NewMetaBuildContextWithSctx(
mock.NewContext(),
metabuild.WithSuppressTooLongIndexErr(true),
)
_, err = ddl.BuildTableInfoWithStmt(ctx, stmt, "utf8mb4", "utf8mb4_bin", nil)
```
This reproduces on TiDB commit `3e4ee9fc99e7`.
### 2. What did you expect to see?
`BuildTableInfoWithStmt` should either:
1. normalize raw `ast.ConstraintFulltext` into the internal columnar FULLTEXT representation and build the corresponding `TableInfo`; or
2. provide/reuse an explicit lightweight normalization API that direct schema-builder callers can invoke without running the complete planner preprocessor.
The exported builder and schema-tracker paths should not treat FULLTEXT as an ordinary KV index on `TEXT`.
### 3. What did you see instead?
The parser leaves the constraint as raw `ast.ConstraintFulltext`. The planner preprocessor normally rewrites it to `ast.ConstraintColumnar` with `ast.IndexTypeFulltext`, but direct builder callers do not run planner preprocessing.
`BuildTableInfoWithStmt` consequently handles the raw constraint as an ordinary index and returns:
```text
[ddl:1170] BLOB/TEXT column 'text_col' used in key specification without a key length
```
This affects TiDB's lightweight schema-builder consumers. In TiFlow DM there are two affected paths:
- source schema tracker bootstrap;
- downstream `TableInfo` cache initialization from `SHOW CREATE TABLE`.
See pingcap/tiflow#12824 for the end-user failure (DM errors 44003 and 44018).
### 4. Root cause and suggested complete fix
FULLTEXT support added a planner preprocessing contract: raw `ConstraintFulltext` is rewritten before metadata construction. `BuildTableInfoWithStmt` remains callable with parser output, and TiDB's DM schema tracker also invokes it directly, so this contract is currently easy to violate.
A complete fix should share FULLTEXT normalization between planner preprocessing and direct metadata-building paths, preserving the same grammar/index-option validation. It should cover the FULLTEXT DDL lifecycle rather than merely dropping metadata, including CREATE/ALTER/DROP behavior where applicable.
A narrow downstream mitigation in TiFlow can ignore raw FULLTEXT constraints because they are non-unique and not used as DML row handles, but that is intentionally not the complete semantic solution.
Related historical changes:
- pingcap/tidb#3658
- pingcap/tidb#9821
- pingcap/tidb#60720
- pingcap/tiflow#12824
### 5. What is your TiDB version?
```text
3e4ee9fc99e7
```
Contributor guide
Research direction
Start at the exported ddl.BuildTableInfoWithStmt entry point and compare its handling of parser.New output with the planner preprocessing that rewrites ast.ConstraintFulltext. Trace the CREATE/ALTER/DROP FULLTEXT paths and the affected schema-builder consumers; done means direct parser ASTs preserve FULLTEXT validation and metadata semantics without the TEXT key-length error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100