pingcap / pingcap/tidb

ddl: support CREATE TABLE ... LIKE ... EXCLUDE PARTITIONS to create non-partitioned copy

Open
#67,897 0 comments 0 reactions 0 assignees View on GitHub
contribution
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Summary

Support `CREATE TABLE temp LIKE src EXCLUDE PARTITIONS` to create a non-partitioned copy of a partitioned table in a single DDL statement, eliminating the need for a follow-up `ALTER TABLE ... REMOVE PARTITIONING`.

## Motivation

TiDB Lightning ETL pipelines ingest into a temp table before `EXCHANGE PARTITION`. Currently this requires two steps:

1. `CREATE TABLE temp LIKE src` — copies partition definitions
2. `ALTER TABLE temp REMOVE PARTITIONING` — expensive full-table rebuild

The two-step pattern causes severe region orphan accumulation in production:

| Metric | Value |
|--------|-------|
| Total regions in cluster | 948,177 |
| Orphan (empty, NULL table) regions | **941,404 (99.3%)** |
| TiKV RSS per node | 34–38 GiB |

Each ETL run inherits ~150 partition definitions and pre-splits ~600 regions, which become orphans after `REMOVE PARTITIONING`. At 1,483 runs this produces ~889,000 orphan regions, causing:

- **Node recovery time:** 14+ hours vs 6 minutes without orphans (140× difference)
- **PD scheduling saturation:** merge checker ran 170M+ times with zero progress
- **Memory pressure:** orphan Raft peers account for the majority of TiKV RSS

## Proposed Change

Add `EXCLUDE PARTITIONS` as an optional clause to `CREATE TABLE ... LIKE`. When present, the produced table has `TableInfo.Partition = nil` — identical to a table defined without any `PARTITION BY` clause.

```sql
CREATE TABLE temp LIKE src EXCLUDE PARTITIONS;
```

## Implementation

- `pkg/parser/parser.y`: new `ExcludePartitionsOpt` production rule; `EXCLUDE` as unreserved keyword
- `pkg/parser/misc.go`: `"EXCLUDE"` in `tokenMap`
- `pkg/parser/ast/ddl.go`: `ExcludePartitions bool` on `CreateTableStmt`; `Restore()` emits the clause
- `pkg/ddl/create_table.go`: `BuildTableInfoWithLike` clears `tblInfo.Partition` and demotes global indexes when flag is set

Suggested labels: type/feature, component/ddl

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.