SovereignCloudStack / SovereignCloudStack/standards

[Feature Request] Introduce testcase-specific reporting

Open
#1,081 4 comments 0 reactions 1 assignee View on GitHub

@mbuechse is already working on this.

Since Jan 30, 2026.

  • #1073 by @mbuechse — closed without merging
enhancement
Dominant language
Python
Stars
47
Forks
38
Avg merge
1d 16h
Merged PRs (30d)
8

Description

Currently, our report assigns to every testcase a result (PASS/FAIL/ABORT). Everything else is buried in blanket stdout/stderr capture, which has its own internal structure, but that is only apparent if you see all the intervening debug messages, which are redacted on the compliance monitor. It's not great UX.

Also stderr should not be relevant to anyone except test developers! That need not even be part of the UI (except for the full report view).

  • specify comprehensive json/yaml format for testcase-specific reporting
  • change all scripts so that report is printed on stdout
  • make sure stderr only contains debug info and unexpected operational errors
  • hide stderr from CM UI (except in full report view)
  • make testcase-specific report available in CM UI (no longer calling it stdout)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.