apple / apple/foundationdb

Storage Server recruitment should consider existing recruited roles

Open
#552 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

If one creates a FDB cluster without specifying process classes, then all processes will have a storage server recruited on them. This is unideal for a high performance setup, as transaction logs will be competing against (one or more) storage servers for disk bandwidth and syncs, proxies and resolvers will compete against storage servers for CPU time, and multiple storage servers will compete with themselves for disk bandwidth and space.

There is a previous attempt at solving this problem in the codebase, that of `configure auto`, which will automatically try to assign process classes to existing processes. But this isn't a comprehensive solution, as it doesn't take process failures into account.

It'd be good to improve storage server recruitment to try and avoid other recruited roles if possible, but still recruit from the avoided processes if required to have enough storage servers to meet the replication requirement, or if the bottleneck of FDB is identified to be the lack of storage servers.

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing `configure auto` process-class assignment and the storage server recruitment path, then trace how process failures and replication requirements affect role selection. Done means recruitment avoids processes hosting other recruited roles when possible, but still selects them when needed for replication or when more storage servers are required.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.