pingcap / pingcap/tidb

Introduce globally restorable physical table identity to avoid BR table ID rewrite

Open
#69,923 1 comment 0 reactions 0 assignees View on GitHub
type/new-feature
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Feature Request

Introduce a globally restorable physical storage identity for TiDB tables and partitions, so that supported BR restore and TiDB-X tenant migration flows can preserve physical data key prefixes instead of rewriting source physical table IDs to target-local physical table IDs.

Detailed RFC: https://github.com/pingcap/tidb/pull/69924

### Problem

TiDB currently encodes the physical table ID directly in row/index key prefixes:

```text
record key: t{physical_table_id}_r{handle}
index key : t{physical_table_id}_i{index_id}...
```

When BR restores a table into another cluster, the target table usually gets different physical table IDs, so BR has to maintain an old-to-new physical table ID mapping and rewrite key prefixes during restore.

This also makes TiDB-X tenant migration harder because table data is tied to target-local `int64` physical table IDs.

### Requested Behavior

Introduce a physical storage identity that can be preserved across backup/restore and tenant migration.

At a high level:

- keep existing `TableInfo.ID` / `PartitionDefinition.ID` for compatibility;
- add a physical storage UUID for tables and partitions;
- introduce a versioned key format that can encode the physical storage UUID in row/index key prefixes;
- let BR preserve UUID-based physical storage identities and restore supported tables without table ID key-prefix rewrite;
- let TiDB-X tenant migration preserve physical data identity across tenants/clusters.

### Alternatives Considered

- Preserve source `int64` physical table IDs for full restore into an empty target cluster. This is useful but limited and does not solve partial restore or tenant migration conflicts.
- Add logical UUID metadata only. This helps lineage and diagnostics, but cannot remove key rewrite while data keys still use `t{int64}` prefixes.
- Allocate globally unique `int64` physical IDs. This keeps keys compact but creates cluster/tenant ID lifecycle and ownership-transfer problems.

### Adoption / Migration Notes

The RFC proposes a phased rollout:

1. metadata preparation;
2. experimental UUID-based table key format;
3. DDL/delete-range coverage;
4. BR no-rewrite path for UUID-format tables;
5. TiDB-X tenant migration POC;
6. ecosystem support for TiFlash, TiCDC, Lightning, statistics, diagnostics, placement, and tooling.

Existing `int64` key-format tables should continue to work and continue using the current BR rewrite path.

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked RFC at https://github.com/pingcap/tidb/pull/69924, then trace the proposed rollout from metadata preparation through the experimental UUID-based key format, DDL/delete-range coverage, and BR integration. Done means supported BR restore and TiDB-X tenant migration preserve physical storage identities without rewriting key prefixes while existing int64-format tables retain their current behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.