adoptium / adoptium/aqa-test-tools
Store Build Configuration Output For Each Benchmark Run
- Dominant language
- Jupyter Notebook
- Stars
- 33
- Forks
- 97
- Avg merge
- 7h 9m
- Merged PRs (30d)
- 5
Description
### Problem Description
Originally, we were running multiple iterations of the benchmarks in one Jenkins build (i.e. Old design). Now, we're moving to a new design in https://github.com/AdoptOpenJDK/openjdk-test-tools/issues/24 in which a parent build would launch multiple child builds, each child build acting as a single benchmark iteration. Hence, we'll be running multiple child jobs instead of one Jenkins builds with multiple iterations inside so that we can interleave Jenkins builds.
https://github.com/AdoptOpenJDK/openjdk-test-tools/blob/eb9c0d302759787e15afe96fe8d24b0e2b1f907c/TestResultSummaryService/parsers/BenchmarkParser.js#L5
Currently, we're just getting the output for all the iterations and not storing any output before the first `********** START OF NEW TESTCI BENCHMARK JOB **********`. This is a bug with the old design since we would only show the the output of the first iteration (`value.tests[0]._id`) when the user clicks on a Jenkins build even though that Jenkins build may have multiple iterations, which won't be displayed. This problem will be fixed with the new design.
https://github.com/AdoptOpenJDK/openjdk-test-tools/blob/eb9c0d302759787e15afe96fe8d24b0e2b1f907c/test-result-summary-client/src/Build/TopLevelBuilds.jsx#L73-L86
### Proposed Changes
We should store the complete output or some useful info before the start of each benchmark (i.e. Before `********** START OF NEW TESTCI BENCHMARK JOB **********` is printed). This would help in having access to the output for the steps that would have been used to download and configure all the benchmark material before the actual benchmark run.
Contributor guide
Research direction
Start with TestResultSummaryService/parsers/BenchmarkParser.js at the referenced line and trace how output before each benchmark marker is parsed and stored. Then inspect test-result-summary-client/src/Build/TopLevelBuilds.jsx lines 73-86 to understand how the stored output is exposed for a Jenkins build. Done means useful or complete configuration output before each benchmark is retained and available with the corresponding build.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, react
- Domain
- full-stack
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100