cockroachdb / cockroachdb/cockroach

catalog/replication: rewrite external descriptor IDs in reader tenant key space

Open
#152,909 6 comments 0 reactions 0 assignees View on GitHub
branch-release-24.3 C-bug T-sql-foundations
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Currently, when PCR sets up the reader tenant's catalog, it naively writes all external descriptors to the same ID. In other words, if table foo has ID 106 in the replicating tenant, PCR will attempt to create the reader's external descriptor foo with ID 106. However, if the reader already contains a different descriptor with ID 106, PCR cannot set up the reader catalog, as it currently cannot handle descriptor ID collisions. There are 2 known ways this descriptor ID collision problem can occur:
- **Source Cluster Created before 22.1**: before 22.2, system table IDs were hardcoded from 1-50, and user tables could be initialized with IDs greater than 50. But, we then needed to create more system tables, so from 22.2 and onward, cluster bootstrap/upgrades would dynamically create system tables with IDs greater than 50. That implies that a source cluster created pre 22.2 may have some user tables with IDs lower than some dynamically created system tables. When the reader tenant on 24.3 is created, however, all of its system tables will be created before the external catalog is initialized with user tables. That implies that for source clusters created pre 22.2, there's ample opportunity for collision: the replicating tenant likely has a user table ID between 50-100 (e.g. defaultDB), but the reader tenant has already dynamically created a system table with that ID.
- **Txn Retries during system table id allocation**: if the txn which reserves a descriptor ID retries, the second attempt may use a higher ID number. In other words, the replicating tenant is not guaranteed to have identical table IDs for its system tables with IDs greater than 50, compared to the standby. For PCR specifically, this can halt reader catalog setup if the `privileges` table has a mismatched ID, as that is the only externalized system table that also gets a dynamically allocated ID (code r[ef](https://github.com/dt/cockroach/blob/jobfrontier/pkg/sql/catalog/replication/reader_catalog.go#L306)). Note: this will only be a problem if the `privileges` table id on the source tenant collides with a different system table on the destination tenant.

Jira issue: CRDB-54107

Epic CRDB-2454

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.