Unit tests of GRV requests
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
When a cluster (in real environment or simulation) does not work as expected, it takes time to identify which component causes the problem.
Since GRV is a critical part in transaction's correctness and performance, we should consider adding unit tests to check its contract.
**Correctness**
- [ ] GRV should monotonically increase, even in different failure scenarios (which will be described later). A test workload can have multiple clients issues GRV and check that the versions monotonically increase per client and across clients;
**Performance**
- [ ] GRV latency should be similar for each client from each proxy;
- [ ] GRV throughput is expected;
- [ ] GRV performance does not degrade much (which will be quantified) when partial failure happens.
**Partial failure**: Failure that does not trigger master recovery.
- Network between a proxy and master or resolver is slinky. The latency on these links is higher;
- A proxy has noisy neighbor and it gets less CPU, cache and memory bandwidth resource;
If only one proxy has the partial failure, an ideal system should redirect traffic to other healthy proxies. The GRV latency should not degrade much. The GRV throughput should only decrease proportional to the number of degraded proxies.
**This is orthogonal to the failure monitoring project**
This issue focuses on testing and understanding if the GRV contract is uphold and how the system's GRV requests reacts to failures.
cc. @sfc-gh-kmakino @sears @yliucode
Contributor guide
Research direction
The issue names no files, existing tests, or specific entry points. Start by locating the GRV request path and test harness, then define workloads for monotonic correctness, client and proxy latency, throughput, and partial failures. Done means repeatable tests quantify the GRV contract and performance behavior under the listed failure scenarios.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- distributed-systems, performance, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100