uutils / uutils/coreutils

CI: add report for tests results (Markdown format)

Open
#5,879 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
24.1k
Forks
2k
Avg merge
1d 5h
Merged PRs (30d)
365

Description

GitHub workflow allows to add a Job summary with Markdown format:

In GH workflows, tests are runned with cargo-nextest tool (instead of cargo test). With version 0.9.66 and above, it adds an experimental support for producing machine-readable output for test runs, in a format similar to cargo test's libtest JSON output => https://nexte.st/book/run-machine-readable.html

I tried to use this feature to generate file results for tests (JSON format) and to use them to generate summaries after end of the tests in FreeBSD workflow (could also be added in other workflows including tests).

  • In "Tests" job, use cargo-nextest to generate results files (JSON):
NEXTEST_EXPERIMENTAL_LIBTEST_JSON=1 cargo nextest run --hide-progress-bar --profile ci --features "${FEATURES}" --message-format libtest-json-plus 1> "${WORKSPACE}/tests-unix.json"

NEXTEST_EXPERIMENTAL_LIBTEST_JSON=1 cargo nextest run --hide-progress-bar --profile ci --all-features -p uucore --message-format libtest-json-plus 1> "${WORKSPACE}/tests-uucore.json"
  • JSON files are copied back from VM to runner/host (copyback: true parameter for the FreeBSD VM action).

  • Add a new step at the end of "Tests" job to parse JSON files and generate summaries (Markdown format)

You could see an example of these summaries in my own repository ("ugly" main branch with a modified FreeBSD workflow, issue #5797 for details) => https://github.com/lcheylus/rust-coreutils/actions/runs/7653519450

I need to do additional tests when some tests are NOK/FAILED and include them in the summary.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the FreeBSD workflow's Tests job and the cargo-nextest commands described in the issue. Check how JSON result files are copied back from the VM, then add the final summary step and validate it with both passing and failed tests. Done means the workflow publishes a Markdown job summary that includes failed-test details.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, rust
Domain
ci-cd, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.