cockroachdb / cockroachdb/cockroach
sql/multiregion: add per-super-region primary and secondary region support
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
## Summary
Add support for declaring a primary and secondary region on a per-super-region basis. The per-super-region primary controls leaseholder affinity for tables homed to that super region. The secondary is the failover target: if the primary goes down, leaseholders move there rather than to an arbitrary member region, staying within the super region boundary.
See the [Super Regions GA design doc](https://docs.google.com/document/d/1US0GzGwqc2flTW81qek0zM_HcaXziAT3HOLhuvoQV_0) for full context.
## Background
A database-level `PRIMARY REGION` only covers leaseholder affinity for the super region it belongs to. Every other super region is left without a leaseholder preference — CRDB falls back to the first member region alphabetically. Similarly, if the database-level `SECONDARY REGION` is in a different super region than the primary, it is a no-op for data in the primary's super region. Per-super-region primary and secondary regions close both gaps.
## SQL Syntax
```sql
CREATE TABLE ( ... )
LOCALITY REGIONAL BY TABLE IN SUPER REGION ;
```
Constraints:
• The super region must exist in the database
```sql
ALTER TABLE SET LOCALITY REGIONAL BY TABLE IN SUPER REGION ;
```
Constraints:
• The super region must exist in the database
```sql
-- Inline at super region creation
ALTER DATABASE mydb ADD SUPER REGION "usa"
VALUES "us-east1", "us-west1", "us-central1"
PRIMARY REGION "us-east1"
SECONDARY REGION "us-west1";
-- As separate statements
ALTER DATABASE mydb ALTER SUPER REGION "usa" SET PRIMARY REGION "us-east1";
ALTER DATABASE mydb ALTER SUPER REGION "usa" SET SECONDARY REGION "us-west1";
-- Also supported inline in CREATE DATABASE
CREATE DATABASE mydb
PRIMARY REGION "us-east1"
REGIONS "us-east1", "us-west1", "us-central1"
SUPER REGION "usa"
VALUES "us-east1", "us-west1", "us-central1"
PRIMARY REGION "us-east1"
SECONDARY REGION "us-west1";
```
Constraints:
- Primary and secondary must be members of the super region — setting either to a non-member is a hard error
- Both are optional
```sql
ALTER DATABASE mydb ALTER SUPER REGION "usa" SET PRIMARY REGION "eu-north1";
-- ERROR: eu-north1 is not a member of super region "usa"
```
## Tasks
- [ ] **SQL parsing changes** — grammar and parser changes to accept `PRIMARY REGION` and `SECONDARY REGION` in `ADD SUPER REGION`, `ALTER SUPER REGION`, and `CREATE DATABASE` syntax. Map all paths to an unimplemented error for now.
- [ ] **Descriptor + schema changer changes** — add new fields to the super region proto/descriptor to store the primary and secondary region. Update all DDL paths (`ADD SUPER REGION`, `ALTER SUPER REGION SET PRIMARY/SECONDARY REGION`, `CREATE DATABASE`) to persist these fields via the schema changer. These are likely best combined into one PR since the descriptor change alone has no observable effect.
- [ ] **Update `SHOW SUPER REGIONS`** — include the primary and secondary region in the output alongside the super region name and member regions.
- [ ] **`REGIONAL BY TABLE IN SUPER REGION` — schema changer + descriptor** — add the new locality type end-to-end: grammar, table descriptor changes, and schema changer support for `CREATE TABLE ... LOCALITY REGIONAL BY TABLE IN SUPER REGION` and `ALTER TABLE ... SET LOCALITY REGIONAL BY TABLE IN SUPER REGION`. This is a prerequisite for the zone config work to be fully testable and meaningful.
- [ ] **Warning when no super region primary is set** — when a table is created or altered with `REGIONAL BY TABLE IN SUPER REGION` and the target super region has no primary region set, emit a `NOTICE` indicating which region was selected as the leaseholder (first member region alphabetically) and a `HINT` on how to set an explicit primary. This fires on both `CREATE TABLE` and `ALTER TABLE ... SET LOCALITY`.
```sql
CREATE TABLE orders (...) LOCALITY REGIONAL BY TABLE IN SUPER REGION 'usa';
NOTICE: super region "usa" has no primary region set. Region "us-central1" was
selected as the leaseholder region (first member region alphabetically).
HINT: To control leaseholder placement, set an explicit primary:
ALTER DATABASE movr ALTER SUPER REGION "usa" SET PRIMARY REGION ;
```
- [ ] **Zone config changes — primary leaseholder preference** — update the span config synthesizer to read the super region's primary from the descriptor and emit the correct lease preference (e.g. `[+region=us-east1]`) for tables homed to that super region via `REGIONAL BY TABLE IN SUPER REGION`. This is the same lease preference mechanism used for `REGIONAL BY TABLE` today. Until this lands, the stored primary field is inert and leaseholder placement falls back to the first member region alphabetically. Note: this task has a soft dependency on `REGIONAL BY TABLE IN SUPER REGION` support (above) to be end-to-end testable.
- [ ] **Zone config changes — secondary failover** — extend the span config synthesizer to emit a second ordered lease preference (`[+region=]`) when the super region has a secondary set. Failover is handled automatically by the lease acquisition algorithm: it tries the first preference (primary), and if unavailable, falls back to the second (secondary). Since both must be super region members, failover stays within the super region boundary. No additional failover infrastructure is needed beyond the ordered lease preferences. This task depends on the primary zone config work above.
## Warnings
- Setting a non-member as primary or secondary is a hard error, not a warning.
- The warning for `REGIONAL BY TABLE IN SUPER REGION` when no super region primary is set is included in this issue (see task above). This also naturally covers the case where the database primary region is not in any super region — the footgun only matters at table placement time, and this notice catches it at exactly the right moment.
- The warning for the database-level secondary being in a different super region than the primary is tracked in #163497.
## Out of Scope
- Per-super-region secondary region support is a run-at for GA and will only be included if time permits.
Epic CRDB-62586
Contributor guide
Assessment
This issue has not been assessed yet.