adoptium / adoptium/aqa-test-tools

Use Common Benchmark Metric Router Design for All Perf Tools

Open
#151 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Jupyter Notebook
Stars
33
Forks
97
Avg merge
7h 9m
Merged PRs (30d)
5

Description

### Problem Description

Currently, TRSS has 2 parsing mechanisms for extract benchmark metric values.

First mechanism is used for Jenkins, which is used by PerfNext and Adopt builds, to parse and store results automatically when a build finishes.

Second mechanism is used for Axxon schedular, which is used by Perffarm currently, to extract numbers from a CSV results file that's generated by Perffarm. Some files related to this mechanism are also used in all frontend perf tools such as Tabular View and Perf Compare.

Maintaining 2 mechanism adds too much redundancy, making it harder to maintain and support different tools.

### Proposed Changes

- Unify both the parsing mechanisms.
- Add `higherbetter` and `units` to [BenchmarkMetric](https://github.com/AdoptOpenJDK/openjdk-test-tools/blob/master/TestResultSummaryService/parsers/BenchmarkMetric.js). Currently, BenchmarkMetric doesn't have that information and we rely on [BenchmarkVariants](https://github.com/AdoptOpenJDK/openjdk-test-tools/blob/master/test-result-summary-client/src/PerfCompare/lib/benchmarkVariantsInfo.js) for it.
- Get rid of BenchmarkVariants file.
- Make an API so that all perf tools can request the backend for the BenchmarkMetric file and use it accordingly.
- Update perf tools to use that API to fetch BenchmarkMetric file instead of using BenchmarkVariants file.

### Assigned Contributors
I'll be working with Dong (@dhlee49) from my team to work on this design.

Contributor guide

Open the contributing guide

Research direction

Start with parsers/BenchmarkMetric.js and test-result-summary-client/src/PerfCompare/lib/benchmarkVariantsInfo.js, then trace the Jenkins and Axxon parsing paths and the frontend performance tools that consume them. Done means the mechanisms share one BenchmarkMetric representation and the perf tools obtain it through the proposed API instead of BenchmarkVariants.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
backend-api-design, frontend, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.