pingcap / pingcap/tidb

The restriction for continuity of DXF add index is too hard

Open
#69,626 1 comment 0 reactions 0 assignees View on GitHub
affects-8.1 affects-8.5 component/ddl severity/major type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

1. Enable distributed DDL backfill for `ADD INDEX`.
2. Create a large table that spans many TiKV regions.
3. Execute `ALTER TABLE ... ADD INDEX ...` so that TiDB uses DXF backfill planning.
4. During backfill subtask planning, let TiKV region metadata change, for example by region merge/split or scheduling.
5. TiDB might get region metadata that is not exactly continuous and repeatedly block or fail the DDL planning.

### 2. What did you expect to see? (Required)

`ADD INDEX` should continue to generate DXF backfill subtasks. Region metadata is only used to roughly split the backfill key range into batches, so transient non-continuous region metadata should not block the DDL as long
as the generated backfill ranges still cover the original table range continuously.

### 3. What did you see instead (Required)

`ADD INDEX` could be blocked during DXF backfill range planning because TiDB required adjacent region metadata to be exactly continuous. In a large table, region merge/split or scheduling might make the region metadata
returned by RegionCache temporarily non-continuous or overlapping.

In one case, an `ADD INDEX` DDL on a large table ran for more than 26 hours before being manually canceled.

### 4. What is your TiDB version? (Required)
v8.5.4

Contributor guide

Open the contributing guide

Research direction

Start at the DXF backfill range-planning path for ALTER TABLE ... ADD INDEX and inspect how RegionCache metadata is handled during region merge, split, or scheduling. Reproduce the large-table scenario with distributed DDL backfill enabled. Done means transient non-contiguous or overlapping metadata no longer blocks planning when generated ranges still cover the original table range continuously.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.