adoptium / adoptium/aqa-test-tools
Use Common Benchmark Metric Router Design for All Perf Tools
- Dominant language
- Jupyter Notebook
- Stars
- 33
- Forks
- 97
- Avg merge
- 7h 9m
- Merged PRs (30d)
- 5
Description
### Problem Description
Currently, TRSS has 2 parsing mechanisms for extract benchmark metric values.
First mechanism is used for Jenkins, which is used by PerfNext and Adopt builds, to parse and store results automatically when a build finishes.
Second mechanism is used for Axxon schedular, which is used by Perffarm currently, to extract numbers from a CSV results file that's generated by Perffarm. Some files related to this mechanism are also used in all frontend perf tools such as Tabular View and Perf Compare.
Maintaining 2 mechanism adds too much redundancy, making it harder to maintain and support different tools.
### Proposed Changes
- Unify both the parsing mechanisms.
- Add `higherbetter` and `units` to [BenchmarkMetric](https://github.com/AdoptOpenJDK/openjdk-test-tools/blob/master/TestResultSummaryService/parsers/BenchmarkMetric.js). Currently, BenchmarkMetric doesn't have that information and we rely on [BenchmarkVariants](https://github.com/AdoptOpenJDK/openjdk-test-tools/blob/master/test-result-summary-client/src/PerfCompare/lib/benchmarkVariantsInfo.js) for it.
- Get rid of BenchmarkVariants file.
- Make an API so that all perf tools can request the backend for the BenchmarkMetric file and use it accordingly.
- Update perf tools to use that API to fetch BenchmarkMetric file instead of using BenchmarkVariants file.
### Assigned Contributors
I'll be working with Dong (@dhlee49) from my team to work on this design.
Contributor guide
Research direction
Start with parsers/BenchmarkMetric.js and test-result-summary-client/src/PerfCompare/lib/benchmarkVariantsInfo.js, then trace the Jenkins and Axxon parsing paths and the frontend performance tools that consume them. Done means the mechanisms share one BenchmarkMetric representation and the perf tools obtain it through the proposed API instead of BenchmarkVariants.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- backend-api-design, frontend, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100