cockroachdb / cockroachdb/cockroach
Consider more aggressively prioritising constraint violations in the kvserver
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Currently, if you spin up a multi-region Standard (and I'm guessing Basic) cluster, there's a chance that additional regions will be present in the cluster (as symptom of splitting off the keyspan and inheriting the prior ranges placement). This can cause additional read and write latency against a user's local region, as data may temporarily exist in a distance region.
Here's an example (note that the behaviour doesn't always present) assuming the following 3 regions on AWS:
* us-east-1
* eu-central-1
* ap-southeast-1
Create a table and insert data:
```sql
CREATE TABLE account (
"id" UUID PRIMARY KEY DEFAULT gen_random_uuid(),
"balance" DECIMAL NOT NULL
) LOCALITY REGIONAL BY ROW;
INSERT INTO account ("id", "balance", "crdb_region")
SELECT
gen_random_uuid(),
ROUND(RANDOM() * 1000, 2),
region::crdb_internal_region
FROM (
SELECT
ARRAY['aws-us-east-1', 'aws-eu-central-1', 'aws-ap-southeast-1'][FLOOR(1 + RANDOM() * 3)::int] AS region
FROM generate_series(1, 1000)
) AS random_regions;
```
Create super regions:
```sql
SET enable_super_regions = 'on';
ALTER DATABASE demo ADD SUPER REGION us VALUES "aws-us-east-1";
ALTER DATABASE demo ADD SUPER REGION eu VALUES "aws-eu-central-1";
ALTER DATABASE demo ADD SUPER REGION ap VALUES "aws-ap-southeast-1";
```
Run the following command to show replicas for one of each region:
```sql
SELECT DISTINCT
split_part(unnest(replica_localities), ',', 1) AS replica_localities,
replicas
FROM [SHOW RANGE FROM TABLE account FOR ROW (
'aws-us-east-1',
(SELECT id FROM account WHERE crdb_region = 'aws-us-east-1' LIMIT 1)
)]
UNION ALL
SELECT DISTINCT
split_part(unnest(replica_localities), ',', 1) AS replica_localities,
replicas
FROM [SHOW RANGE FROM TABLE account FOR ROW (
'aws-eu-central-1',
(SELECT id FROM account WHERE crdb_region = 'aws-eu-central-1' LIMIT 1)
)]
UNION ALL
SELECT DISTINCT
split_part(unnest(replica_localities), ',', 1) AS replica_localities,
replicas
FROM [SHOW RANGE FROM TABLE account FOR ROW (
'aws-ap-southeast-1',
(SELECT id FROM account WHERE crdb_region = 'aws-ap-southeast-1' LIMIT 1)
)];
replica_localities | replicas
----------------------------+-------------
region=aws-us-east-1 | {3,29,32}
region=aws-eu-central-1 | {12,16,24}
region=aws-us-west-2 | {12,16,24}
region=aws-ap-southeast-1 | {6,7,16}
region=aws-eu-central-1 | {6,7,16}
```
Notice the additional us-west-2 region and the duplication of replicas {12,16,24} and {6,7,16}. This doesn't always occur and always resolves itself when it does but may take around 10 minutes (which is find for production but annoying for demos).
One possible solution (suggested by Austen) would be to more aggressively prioritise constraint violations in the kvserver by upgrading the priority of constraint violations (they are the same as rebalancing atm, since they are rebalancing, the range has the right number of replicas).
Jira issue: CRDB-42984
Contributor guide
Assessment
This issue has not been assessed yet.