apple / apple/foundationdb

Coordinator fault tolerance calculation depends on coordinators being part of the cluster

Open
#4,833 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

The fault tolerance calculation includes a check to make sure that our coordinators are distributed across sufficient fault tolerance zones. This check relies on the coordinators being part of the cluster in order to determine the coordinator locality.

In the event that you are using coordinators that aren't part of the cluster, the locality check assigns each coordinator to the empty zone. As a result, the check believes that the loss of this one zone would bring down your cluster and assigns a fault tolerance of 0.

See https://forums.foundationdb.org/t/fault-tolerance-0-zones-after-setting-locality-zoneid/2706/8.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the fault-tolerance calculation and coordinator locality handling described in the issue. Reproduce the case with coordinators that are not part of the cluster and verify that locality no longer places them all in the empty zone and that the reported fault tolerance reflects the configured fault-tolerance zones.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
40/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.