hiero-ledger / hiero-ledger/hiero-consensus-node

JMH tests history and comparison automation

Open
#19,045 1 comment 0 reactions 1 assignee Assigned to @alex-kuzmin-hg View on GitHub
DevOps Performance Test Development
Dominant language
Java
Stars
406
Forks
226
Avg merge
3d 4h
Merged PRs (30d)
210

Description

### Background

Currently we don't have an option to compare JMH test results with previous runs.

It would be nice to have periodic job that runs high-level JMH tests and compares them with previous runs.
Additionally (could be follow-up task), would be nice for a developer to run such workflow manually against custom branch to compare results with base branch (main or release), to ensure changes in the branch are not impacting the performance.

Comparison output could be as a simple table (like coverage diff we have now), but in future some percentile in degradation can be introduced to create automatic alerts.

### Acceptance Criteria

- [ ] Create a GitHub workflow that allows running JMH tests from `swirlds-benchmarks` module, outputting results into a json file, and comparing results with previous runs. Comparison should identify and output performance degradation for the same tests. Keeping an output file history is required for comparison.
- [ ] The workflow above should be configured to run periodically against the `main` or release branches and notify about performance degradation.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.