pingcap / pingcap/tidb

ddl: integrate DROP/TRUNCATE PARTITION global index cleanup into DXF

Open
#65,418 0 comments 0 reactions 0 assignees View on GitHub
type/new-feature
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Problem

`DROP PARTITION` / `TRUNCATE TABLE PARTITION` needs to clean up **global index** entries for old partitions during the `StateDeleteReorganization` phase. The current implementation executes through the DDL owner's local reorg/backfill worker (single-node bottleneck).

The goal is to integrate this cleanup into DXF (Distributed Task Framework), enabling scan/delete operations to run in parallel across TiDB nodes while maintaining pause/resume capability and correctness.

**Note**: Once cleanup execution starts, cancel is not supported.

**Scope**: Covers global index cleanup for `DROP PARTITION` and `TRUNCATE TABLE PARTITION`; other DDLs are out of scope.

## Current State

- **Trigger point**: Both `onDropTablePartition` and `onTruncateTablePartition` call `cleanGlobalIndexEntriesFromDroppedPartitions` when in `StateDeleteReorganization` and the table has global indexes.
- **Current cleanup implementation**: `cleanGlobalIndexEntriesFromDroppedPartitions` constructs `reorgInfo` via `getReorgInfoFromPartitions(...)`, then enters the local reorg execution path through `runReorgJob(..., w.cleanupGlobalIndexes(...))`.
- **Delete semantics**: `cleanUpIndexWorker` scans rows from old partitions, generates global index keys, uses `BatchGet` to find current index entries, and only locks and deletes when the entry value decodes to `kv.PartitionHandle` with `PartitionID == scanned partition ID`, avoiding accidental deletion of entries written by new/other partitions.

## Proposed Changes

### 1. Add new DXF TaskType: GlobalIndexCleanup
- Add new `TaskType` in `pkg/disttask/framework/proto/type.go`
- Add step constant in `pkg/disttask/framework/proto/step.go`

### 2. Task Meta and Subtask Definition
- Define `CleanupTaskMeta` (JSON):
- `Job model.Job` (clone, includes SchemaID/TableID/JobID/Priority/ReorgMeta, etc.)
- `TableInfo *model.TableInfo` (must allow `table.TableFromMeta` to locate old partition physical tables via `OldPartitionIDs`)
- `OldPartitionIDs []int64` (dropping partition IDs for DROP or old partition IDs for TRUNCATE)
- `GlobalIndexIDs []int64`
- `Version int` (for compatibility)
- Define `CleanupSubtaskMeta` (JSON):
- `PhysicalTableID int64` (old partition physical ID)
- `RowStart []byte`, `RowEnd []byte` (record key range)

### 3. Scheduler: Generate region-grouped scan ranges for old partitions
- Create `pkg/ddl/globalindexcleanup/scheduler.go`, implement `scheduler.Extension`
- In `OnNextSubtasksBatch` for `CleanupStepScanAndDelete`:
- Read `OldPartitionIDs` from task meta
- For each old partition, reuse add-index scheduler's planning approach:
- Use `getTableRange(...)` to get record key `[start,end]`
- Use `regionCache.LoadRegionsInKeyRange(...)` to get covering regions
- Use `CalculateRegionBatch`-like strategy to batch regions into multiple subtask metas

### 4. Executor: Execute "scan rows -> delete corresponding global index entries" on each subtask
- Create `pkg/ddl/globalindexcleanup/executor.go`, implement `taskexecutor.TaskExecutor`
- Executor logic:
- Construct `table.Table` from task meta's `TableInfo`, locate old partition's `PhysicalTable` by `PhysicalTableID`
- Build global index list (filter indexInfo/index objects by `GlobalIndexIDs`)
- Scan records in `[RowStart, RowEnd)` range, reuse/migrate core delete semantics from `cleanUpIndexWorker`:
- Generate global index key for each row; `BatchGet` to find existing entries
- If entry value decodes to `kv.PartitionHandle` and `PartitionID != current old pid`, skip to avoid accidental deletion
- If matched, `LockKeys` then delete to avoid concurrent write races
- Subtask idempotency: repeated execution only re-deletes non-existent keys or skips non-matching partition entries

### 5. DDL Worker Integration: Submit and wait for DXF task during DROP/TRUNCATE PARTITION reorg phase
- Modify `cleanGlobalIndexEntriesFromDroppedPartitions`: use DXF when distributed conditions are met, otherwise fall back to existing local reorg logic
- DXF branch helper (similar to `executeDistTask`):
- Construct stable task key: `keyspace?/ddl//`
- Check if task already exists and succeeded via `GetTaskByKeyWithHistory`; if exists, resume/wait
- Wait for completion via `WaitTaskDoneOrPaused`
- Periodically aggregate subtask row_count and sync to DDL job.RowCount during wait
- Reuse `checkRunnableOrHandlePauseOrCanceled` pattern to sync DDL job pause/resume to DXF task

### 6. ReorgMeta Support
- Initialize `job.ReorgMeta` for `ActionDropTablePartition` and `ActionTruncateTablePartition`
- Extend `initJobReorgMetaFromVariables` switch to cover these two job types
- Call it after job creation in `executor.DropTablePartition` / `executor.TruncateTablePartition`

### 7. Registration
- Register new task type's executor and scheduler factory in `pkg/ddl/ddl.go`'s `newDDL`

## Tests

### Unit Tests
- **Scheduler plan generation**: `pkg/ddl/globalindexcleanup/scheduler_test.go`
- Only generate subtasks for `OldPartitionIDs` (not visible new partitions)
- Each meta deserializes to `CleanupSubtaskMeta` with correct `PhysicalTableID` and `RowStart < RowEnd`
- Empty partition/range returns 0 subtasks

- **Delete decision logic**: `pkg/ddl/globalindexcleanup/executor_test.go`
- Value is not `kv.PartitionHandle` → don't delete
- `PartitionID != old pid` → don't delete
- Decode error/incompatible format → don't delete and don't panic

### Functional Tests
- **DDL + DXF end-to-end**: `pkg/ddl/tests/partition/db_partition_test.go`
- `DROP PARTITION` + global index
- `TRUNCATE TABLE PARTITION` + global index
- Assertions:
- Explicitly set `tidb_enable_dist_task=ON`, `tidb_ddl_enable_fast_reorg=ON`
- Result set correct after DDL completion
- `checkGlobalIndexCleanUpDone` asserts old partition's global index entries are cleaned
- Query `mysql.tidb_global_task(_history)` to find cleanup task by key prefix `ddl//` with final state succeed

- **Pause/Resume semantics** (optional but recommended)

## Related Code

- `pkg/ddl/partition.go` - cleanGlobalIndexEntriesFromDroppedPartitions
- `pkg/ddl/index.go` - cleanUpIndexWorker
- `pkg/ddl/backfilling_dist_scheduler.go` - reference implementation

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.