adoptium / adoptium/aqa-test-tools

Store Build Configuration Output For Each Benchmark Run

Open
#70 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Jupyter Notebook
Stars
33
Forks
97
Avg merge
7h 9m
Merged PRs (30d)
5

Description

### Problem Description

Originally, we were running multiple iterations of the benchmarks in one Jenkins build (i.e. Old design). Now, we're moving to a new design in https://github.com/AdoptOpenJDK/openjdk-test-tools/issues/24 in which a parent build would launch multiple child builds, each child build acting as a single benchmark iteration. Hence, we'll be running multiple child jobs instead of one Jenkins builds with multiple iterations inside so that we can interleave Jenkins builds.

https://github.com/AdoptOpenJDK/openjdk-test-tools/blob/eb9c0d302759787e15afe96fe8d24b0e2b1f907c/TestResultSummaryService/parsers/BenchmarkParser.js#L5

Currently, we're just getting the output for all the iterations and not storing any output before the first `********** START OF NEW TESTCI BENCHMARK JOB **********`. This is a bug with the old design since we would only show the the output of the first iteration (`value.tests[0]._id`) when the user clicks on a Jenkins build even though that Jenkins build may have multiple iterations, which won't be displayed. This problem will be fixed with the new design.

https://github.com/AdoptOpenJDK/openjdk-test-tools/blob/eb9c0d302759787e15afe96fe8d24b0e2b1f907c/test-result-summary-client/src/Build/TopLevelBuilds.jsx#L73-L86

### Proposed Changes
We should store the complete output or some useful info before the start of each benchmark (i.e. Before `********** START OF NEW TESTCI BENCHMARK JOB **********` is printed). This would help in having access to the output for the steps that would have been used to download and configure all the benchmark material before the actual benchmark run.

Contributor guide

Open the contributing guide

Research direction

Start with TestResultSummaryService/parsers/BenchmarkParser.js at the referenced line and trace how output before each benchmark marker is parsed and stored. Then inspect test-result-summary-client/src/Build/TopLevelBuilds.jsx lines 73-86 to understand how the stored output is exposed for a Jenkins build. Done means useful or complete configuration output before each benchmark is retained and available with the corresponding build.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, react
Domain
full-stack
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.