support deterministic failover schedule for placement rules
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Enhancement
My deployment scenario involves two "primary" regions in AWS:
- us-east-1
- us-west-2
I have been experimenting with placement rules with a third region: `us-east-2`. This region should only be used for quorum, as there are no application servers hosted in it. So I define a placement policy as follows:
```sql
CREATE PLACEMENT POLICY `defaultpolicy` PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-2,us-east-2";
```
Because the pd-server supports a `weight` concept, when `us-east-1` fails I can have the pd-leader be deterministic and `us-west-2` will become the leader. However, there is no deterministic behavior of where the leader of regions for `defaultpolicy` will go. They will likely balance across `us-west-2` and `us-east-2`, which is not the desired behavior.
Ideally I want the priority for the leader to be in-order of the region-list. This means that `us-west-2` will become the new leader for all regions. Perhaps this could be conveyed with syntax like:
```sql
CREATE PLACEMENT POLICY `defaultpolicy` PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-2,us-east-2" SCHEDULE="DETERMINISTIC";
```
In fact, if this worked deterministically for leader-scheduling and follower-scheduling, and extension of this is I could create the following:
```sql
CREATE PLACEMENT POLICY `defaultpolicy` PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-2,us-east-2,us-west-1" SCHEDULE="DETERMINISTIC";
```
Since the default followers is 2, it would mean that `us-west-1` won't get regions scheduled _unless_ one of the other regions fails, which suits me perfectly. It will also mean that commit latency is only initially bad when failover to `us-west-2` first occurs. Over time as regions as migrated to `us-west-1`, the performance should be ~restored as quorum can be achieved on the west coast.
This is a really common deployment pattern in the continental USA, so I'm hoping it can be implemented :-)
Contributor guide
Assessment
This issue has not been assessed yet.