hiero-ledger / hiero-ledger/hiero-consensus-node
JMH tests history and comparison automation
- Dominant language
- Java
- Stars
- 406
- Forks
- 226
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 210
Description
### Background
Currently we don't have an option to compare JMH test results with previous runs.
It would be nice to have periodic job that runs high-level JMH tests and compares them with previous runs.
Additionally (could be follow-up task), would be nice for a developer to run such workflow manually against custom branch to compare results with base branch (main or release), to ensure changes in the branch are not impacting the performance.
Comparison output could be as a simple table (like coverage diff we have now), but in future some percentile in degradation can be introduced to create automatic alerts.
### Acceptance Criteria
- [ ] Create a GitHub workflow that allows running JMH tests from `swirlds-benchmarks` module, outputting results into a json file, and comparing results with previous runs. Comparison should identify and output performance degradation for the same tests. Keeping an output file history is required for comparison.
- [ ] The workflow above should be configured to run periodically against the `main` or release branches and notify about performance degradation.
Contributor guide
Assessment
This issue has not been assessed yet.