Introduce globally restorable physical table identity to avoid BR table ID rewrite
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Feature Request
Introduce a globally restorable physical storage identity for TiDB tables and partitions, so that supported BR restore and TiDB-X tenant migration flows can preserve physical data key prefixes instead of rewriting source physical table IDs to target-local physical table IDs.
Detailed RFC: https://github.com/pingcap/tidb/pull/69924
### Problem
TiDB currently encodes the physical table ID directly in row/index key prefixes:
```text
record key: t{physical_table_id}_r{handle}
index key : t{physical_table_id}_i{index_id}...
```
When BR restores a table into another cluster, the target table usually gets different physical table IDs, so BR has to maintain an old-to-new physical table ID mapping and rewrite key prefixes during restore.
This also makes TiDB-X tenant migration harder because table data is tied to target-local `int64` physical table IDs.
### Requested Behavior
Introduce a physical storage identity that can be preserved across backup/restore and tenant migration.
At a high level:
- keep existing `TableInfo.ID` / `PartitionDefinition.ID` for compatibility;
- add a physical storage UUID for tables and partitions;
- introduce a versioned key format that can encode the physical storage UUID in row/index key prefixes;
- let BR preserve UUID-based physical storage identities and restore supported tables without table ID key-prefix rewrite;
- let TiDB-X tenant migration preserve physical data identity across tenants/clusters.
### Alternatives Considered
- Preserve source `int64` physical table IDs for full restore into an empty target cluster. This is useful but limited and does not solve partial restore or tenant migration conflicts.
- Add logical UUID metadata only. This helps lineage and diagnostics, but cannot remove key rewrite while data keys still use `t{int64}` prefixes.
- Allocate globally unique `int64` physical IDs. This keeps keys compact but creates cluster/tenant ID lifecycle and ownership-transfer problems.
### Adoption / Migration Notes
The RFC proposes a phased rollout:
1. metadata preparation;
2. experimental UUID-based table key format;
3. DDL/delete-range coverage;
4. BR no-rewrite path for UUID-format tables;
5. TiDB-X tenant migration POC;
6. ecosystem support for TiFlash, TiCDC, Lightning, statistics, diagnostics, placement, and tooling.
Existing `int64` key-format tables should continue to work and continue using the current BR rewrite path.
Contributor guide
Research direction
Start by reading the linked RFC at https://github.com/pingcap/tidb/pull/69924, then trace the proposed rollout from metadata preparation through the experimental UUID-based key format, DDL/delete-range coverage, and BR integration. Done means supported BR restore and TiDB-X tenant migration preserve physical storage identities without rewriting key prefixes while existing int64-format tables retain their current behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100