SovereignCloudStack / SovereignCloudStack/standards
[Feature Request] Introduce testcase-specific reporting
Open
@mbuechse is already working on this.
Since Jan 30, 2026.
- #1073 by @mbuechse — closed without merging
enhancement
- Dominant language
- Python
- Stars
- 47
- Forks
- 38
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 8
Description
Currently, our report assigns to every testcase a result (PASS/FAIL/ABORT). Everything else is buried in blanket stdout/stderr capture, which has its own internal structure, but that is only apparent if you see all the intervening debug messages, which are redacted on the compliance monitor. It's not great UX.
Also stderr should not be relevant to anyone except test developers! That need not even be part of the UI (except for the full report view).
- specify comprehensive json/yaml format for testcase-specific reporting
- change all scripts so that report is printed on stdout
- make sure stderr only contains debug info and unexpected operational errors
- hide stderr from CM UI (except in full report view)
- make testcase-specific report available in CM UI (no longer calling it stdout)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.