Support anti-host grouping preference for team creation
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
Cloud provider can provide the capability to specify which instances (where fdbserver runs) are _never_ on the same physical host. For example, it can specify a group of 7 VMs will never be on the same host. Let's call it anti-host locality for the simplicity of discussion.
Distributing replicas to instances that do not share the same host can provide extra resilience. FDB can achieve this, if system operators know which instances are on the same physical host, which is known in native setting. However, in the above cloud setting, FDB does not explicitly guarantee that in `triple` replicas setting. Note, FDB implicitly guarantee that if replicas setting is `three_data_hall`.
It will be beneficial to provide the anti-host grouping preference capability for team create. When a team (which can be server team or tLog team) is created, the `selectReplicas()` can prefer choosing the `LocalityData`s that are not on the same host.
Solution:
We can extend the `ReplicationPolicy` (`PolicyOne` class to be precise): When `selectReplica()` is called for `PolicyOne`, it first tries to find a `LocalityData` that is in the same anti-host group. If it fails, it falls back to the current logic.
To ignore the anti-host locality feature, operators just need to not set the anti-host locality entry. This should not have impact on the existing clusters that are configured unaware of this extension.
Acknowledgment
This anti-host locality grouping requirement was originally raised by @mpilman offline.
Contributor guide
Research direction
Start by locating the ReplicationPolicy PolicyOne implementation and the selectReplicas() or selectReplica() paths used when creating server and tLog teams. Read how current replica locality is selected and how locality entries are represented. Done means anti-host grouping is preferred when configured, existing behavior remains when it is absent, and fallback selection still works when the preference cannot be satisfied.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100