To enable baseline comparison for performance optimization PRs, kvrocks needs benchmark and latency testing support
- Dominant language
- C++
- Stars
- 4.4k
- Forks
- 658
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 10
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/kvrocks/issues) and found no similar issues.
### Motivation
I compared the benchmark testing in Redis and kvrocks, and found that kvrocks lacks benchmark testing and latency testing. Here's the comparison table:
| Test Type | Redis | kvrocks |
|-----------|-------|---------|
| Unit Test | ✅ | ✅ |
| Integration Test | ✅ | ✅ |
| Benchmark Test | ✅ redis-benchmark | ❌ |
| Latency Test | ✅ --latency, --latency-history | ❌ |
This makes performance-related PRs difficult to review, since contributors cannot provide performance comparisons (unless they use these tools to test themselves, but this adds burden to
contributors).
Therefore, I believe kvrocks needs to support running these tests locally and provide a comprehensive test suite, allowing contributors to easily run performance tests within the kvrocks
repository.
### Solution
Support benchmark testing via redis-benchmark, and provide latency testing via redis-cli. Meanwhile, migrate the test suites from Redis (only the parts that kvrocks already supports).
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by comparing the Redis benchmark and latency testing tools with the test suites that Kvrocks already supports. Identify how redis-benchmark and redis-cli can run locally, then verify that supported Redis tests cover benchmark and latency scenarios and that contributors can reproduce performance comparisons.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, redis
- Domain
- databases, performance, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100