adoptium / adoptium/aqa-test-tools

EPIC: Improve the contents and organization of release summary report

Open
#649 3 comments 0 reactions 1 assignee Claimed by @smlambert View on GitHub
enhancement epic
Dominant language
Jupyter Notebook
Stars
33
Forks
97
Avg merge
7h 9m
Merged PRs (30d)
5

Description

We need to improve the release summary report to make it easier to navigate by triagers during a release.

- [ ] Organize the report by platform.
The release summary report does not organize all the test failures of the same platform together. Since the purpose of the report is to make it easier for triagers to review and triage and they do that by platform, we should group all results of a single platform together.

- [ ] Put a summary table at the top of the report
Creating a table with a column for triagers to be able to annotate as they verify and work through the triage to mark the platform as completed and ready to release would be good. This should also include the platforms that pass, not just ones with failures in them. Some examples of reformatting the report in this way can be found here: https://github.com/adoptium/aqa-tests/issues/4256 and https://github.com/adoptium/aqa-tests/issues/4258

- [ ] Consider removing some information from the main report.
Some types of tests seem to include the java -version info, others do not and it's quite cluttered when it is included in the main report. We can consider removing this or making it available as a comment at the bottom of the platform section.

- [ ] Consider what rerun information is needed in the summary report as per https://github.com/adoptium/aqa-test-tools/issues/649#issuecomment-1892497472

- [ ] Currently, the release summary report summarizes test targets, when available, we should also aim to include details about test cases for the test suites at supply that information (openjdk, jck) as per https://github.com/adoptium/aqa-test-tools/issues/841

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.