apple / apple/foundationdb

Emulate production workload at larger scales

Open
#2,165 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

The workload used to test FDB's scalability is better to be as similar to a real production workload as possible.

A workload's characteristics include:
1) The statistical distributions of data stored in the disk. For example, how data is skewed. This will affect how shards are created, for example;
2) The read and write pattern;
(and more)

To emulate a production workload, we need:
- [ ] Capture the synopsis of production workload;
- [ ] Capture the workload pattern per FDB component, so that we can just test a single component instead of the entire cluster to reduce the testing cost;

[This idea was proposed and discussed in FDB Offset 2019.]

Contributor guide

Open the contributing guide

Research direction

The issue names no implementation files, tests, or entry points; start by reviewing the FDB Offset 2019 discussion referenced in the issue. Define the production-workload synopsis and per-component workload patterns, and consider the work done when both checklist items are captured for larger-scale scalability testing.

Written by the indexing model from the issue text.

Assessment

Domain
distributed-systems, performance, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.