apple / apple/foundationdb

Two storage servers being recruited on the same process after a failure

Open
#1,594 3 comments 0 reactions 0 assignees View on GitHub
data distribution
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

As part of testing fearless cluster, remote data center was brought down for sometime and when they were back up, there were multiple instances of two storage servers being recruited on the same process in the remote DC. It is not harmful as i believe the second one will go away eventually. But it should be looked into why there was additional storage server being recruited and try to avoid it.

Contributor guide

Open the contributing guide

Research direction

Reproduce the reported fearless-cluster scenario by bringing the remote data center down and back up during testing. Investigate why two storage servers are recruited on the same process after recovery, then verify that only the expected server remains and duplicate recruitment is avoided.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.