pingcap / pingcap/tidb

lightning: pre-created table is not skipped when creating it from the source schema fails

Open
#71,376 2 comments 0 reactions 0 assignees View on GitHub
component/lightning may-affects-25.10 may-affects-26.3 may-affects-26.9 may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/major type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

### 1. Minimal reproduce step (Required)

1. Prepare a dumpling-style source whose table schema file uses a collation TiDB does not support when new collation is enabled, e.g.
```sql
-- e2e.t1-schema.sql
CREATE TABLE `t1` (`id` int NOT NULL, `name` varchar(20), PRIMARY KEY (`id`))
DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci;
```
2. As a workaround, create the table downstream in advance with a supported collation:
```sql
CREATE TABLE e2e.t1 (id int NOT NULL, name varchar(20), PRIMARY KEY (id)) COLLATE=utf8mb4_unicode_ci;
```
3. Import the source with TiDB Lightning (any backend), IMPORT INTO via the import SDK, or DM (which uses the same schema importer).

### 2. What did you expect to see? (Required)

The pre-created table is kept and the data is imported, as it was before #58636 (the schema importer used to list the downstream tables first and skip existing ones).

### 3. What did you see instead (Required)

```
create schema failed, table: `e2e`.`t1`, stmt: import table schema: run create schema job failed:
Error 1273 (HY000): Unsupported collation when new collation is enabled: 'utf8mb4_unicode_520_ci'
```

Since #58636, `SchemaImporter.runCreateTableJob` runs `CREATE TABLE IF NOT EXISTS` and only checks table existence when the statement cannot be parsed. `utf8mb4_unicode_520_ci` parses, and TiDB validates the collation in `BuildTableInfoWithStmt` before it checks whether the table exists, so `IF NOT EXISTS` does not help and there is no way to work around such tables without editing the source schema files.

### 4. What is your TiDB version? (Required)

master (`pkg/lightning/mydump/schema_import.go`); v26.3.x; release-8.5 hotfix branches that include #58636.

Contributor guide

Open the contributing guide

Research direction

Start in pkg/lightning/mydump/schema_import.go, especially SchemaImporter.runCreateTableJob, and trace how existing downstream tables are detected before schema validation. Verify the behavior across the TiDB Lightning, import SDK, and DM paths described in the report. Done means a pre-created table with an unsupported source collation is retained and its data is imported without editing the source schema.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.