apple / apple/foundationdb

Expose team info. and confidence of not losing data to Status

Open
#1,228 0 comments 0 reactions 0 assignees View on GitHub
data distribution
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

The current status has a "fault_tolerance" section to describe the max_machine_failures_without_losing_data, which is the `replica_factor - 1`.
It does NOT describe the chance of not losing data when `replica_factor` machines fail at the same time.

FDB carefully chooses the replicas (though the concept of server and machine teams) so that even when `replica_factor` machines fail at the same time, the chance of losing data is still very small, providing extra confidence in data integrity.

In order for system operators to better understand the chance of not losing data in the rare but catastrophic event, we can expose to status the probability of not losing data when `replica_factor` machines fail at the same time. If possible, we can take one step further to expose the information (4 pairs) of team collection to status.

The proposed new information in Status can help system operators to better layout the processes in a cluster: such as deciding how many processes should be put in the same machine and how many machines should the cluster have for better fault tolerance.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing Status fault_tolerance section and the server and machine team concepts described in the issue. Define how the probability of retaining data and, if feasible, the four team-collection pairs should be represented; done means Status exposes the requested information for operators.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.