Emulate production workload at larger scales
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
The workload used to test FDB's scalability is better to be as similar to a real production workload as possible.
A workload's characteristics include:
1) The statistical distributions of data stored in the disk. For example, how data is skewed. This will affect how shards are created, for example;
2) The read and write pattern;
(and more)
To emulate a production workload, we need:
- [ ] Capture the synopsis of production workload;
- [ ] Capture the workload pattern per FDB component, so that we can just test a single component instead of the entire cluster to reduce the testing cost;
[This idea was proposed and discussed in FDB Offset 2019.]
Contributor guide
Research direction
The issue names no implementation files, tests, or entry points; start by reviewing the FDB Offset 2019 discussion referenced in the issue. Define the production-workload synopsis and per-component workload patterns, and consider the work done when both checklist items are captured for larger-scale scalability testing.
Written by the indexing model from the issue text.
Assessment
- Domain
- distributed-systems, performance, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100