fullstaq-ruby / fullstaq-ruby/umbrella
Benchmark for performance and memory usage
- Dominant language
- No language data
- Stars
- 87
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Issue fullstaq-labs/fullstaq-ruby-server-edition#7 described tuning additional settings for server-friendliness. I think that's a good idea, but one potential caveat is that we tune things wrongly. Of course it's impossible to tune things in a way that it's better for *all* workloads, but at the very least I think we should have some kind of performance and memory usage benchmark. However imperfect such a benchmark may be, having something is better than doing guesswork with no benchmark.
A standardized benchmark suite will also help us compare, when we implement new improvements (like https://github.com/fullstaq-labs/fullstaq-ruby-server-edition/issues/57), or when we research new memory allocators (like #4 and #6).
There are currently third party benchmark reports, but they are erratic and are hard to reproduce, so we can't rely on them.
How would such a benchmark look like? Can we reuse existing benchmarks or do we have to write a new one? Help on building or integrating a benchmark would be much appreciated.
The first goal is to have a benchmark suite. Once we have it, a second goal (which will be codified in a separate issue) is to integrate the benchmark in a CI and monitoring system (https://github.com/fullstaq-labs/fullstaq-ruby-server-edition/issues/58), so that we can automatically see the impact of changes over time.
Contributor guide
Research direction
No files, tests, or entry points are named in the issue. Start by reviewing the referenced server-friendliness, improvement, memory-allocator, and CI issues, then define a reproducible performance and memory benchmark suite; done means the suite exists and can compare changes over time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ruby
- Domain
- performance, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100