apple / apple/foundationdb

Add an API to support splitting a given range into shards individually

Open
#11,510 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

Writing a continuous Key space under heavy pressure can cause hot spots, causing storage_server_write_queue_size or even process_behind. However, such hot spots may be known to the upper layer services but cannot be avoided. I wish there was an API that partitioned the shards ahead of time so that the write load was evenly distributed across multiple storageservers. In this way, the upper business can actively divide the Keys space to reduce hot spots.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing FoundationDB's existing range and shard-management APIs, then trace how storage servers handle write queues and shard placement. Define the API boundary and behavior needed to partition a key range ahead of time, and identify tests that can demonstrate more even distribution without introducing hot spots.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.