fullstaq-ruby / fullstaq-ruby/umbrella

Benchmark for performance and memory usage

Open
#9 3 comments 0 reactions 0 assignees View on GitHub
help wanted
Dominant language
No language data
Stars
87
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Issue fullstaq-labs/fullstaq-ruby-server-edition#7 described tuning additional settings for server-friendliness. I think that's a good idea, but one potential caveat is that we tune things wrongly. Of course it's impossible to tune things in a way that it's better for *all* workloads, but at the very least I think we should have some kind of performance and memory usage benchmark. However imperfect such a benchmark may be, having something is better than doing guesswork with no benchmark.

A standardized benchmark suite will also help us compare, when we implement new improvements (like https://github.com/fullstaq-labs/fullstaq-ruby-server-edition/issues/57), or when we research new memory allocators (like #4 and #6).

There are currently third party benchmark reports, but they are erratic and are hard to reproduce, so we can't rely on them.

How would such a benchmark look like? Can we reuse existing benchmarks or do we have to write a new one? Help on building or integrating a benchmark would be much appreciated.

The first goal is to have a benchmark suite. Once we have it, a second goal (which will be codified in a separate issue) is to integrate the benchmark in a CI and monitoring system (https://github.com/fullstaq-labs/fullstaq-ruby-server-edition/issues/58), so that we can automatically see the impact of changes over time.

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named in the issue. Start by reviewing the referenced server-friendliness, improvement, memory-allocator, and CI issues, then define a reproducible performance and memory benchmark suite; done means the suite exists and can compare changes over time.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
performance, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.