Azure-Samples / Azure-Samples/azuresandbox
Emit machine-readable test results (JSON/JUnit) from Invoke-UnitTests.ps1 for CI/CD
- Dominant language
- PowerShell
- Stars
- 154
- Forks
- 75
- Avg merge
- 15h 2m
- Merged PRs (30d)
- 45
Description
## Summary
Prerequisite for #173 (CD) — specifically for precise, per-check test-failure surfacing in GitHub Actions.
`scripts/Invoke-UnitTests.ps1` today only exposes:
- An **exit code** (`0` pass / `1` test failures / `2` harness error).
- A human-readable **log file** + stdout containing per-check `[PASS]`/`[FAIL] ` lines and a
per-module `[SUMMARY] Passed: N Failed: N Total: N` line.
- A final per-**module** summary table; the orchestrator keeps only per-module counts in memory
(`$moduleResults`) — there is **no machine-readable artifact** and no structured per-check result.
This makes it hard for the CD workflow to pinpoint the exact failing assertion: it would have to
regex-parse free-text log lines (brittle).
## Goal
Emit a **stable, machine-readable results file** so CI/CD can render per-check failures robustly
(GitHub Job Summary table, `::error::` annotations, test-reporter actions).
## Acceptance criteria
- [ ] Optional output path parameter(s) (e.g. `-ResultsJsonPath` and/or `-JUnitXmlPath`) that, when
supplied, write structured results without changing default behavior.
- [ ] **JSON** output capturing, at minimum: overall pass/fail + counts, and per-module results with
a per-**check** breakdown (check name, status, and message/detail for failures).
- [ ] **JUnit XML** output (testsuites/testcase, with `` for failed checks) so standard
GitHub test-reporter actions can consume it.
- [ ] Per-check detail is captured at the orchestrator level (not just per-module counts) — the
per-module test scripts surface individual check results to the orchestrator.
- [ ] Existing exit codes, log file, and console output remain unchanged (backward compatible).
- [ ] README / header comments document the new outputs.
## Notes
- Consumed by the #173 CD workflow's `workflow-tests` step to build a `$GITHUB_STEP_SUMMARY` table
and emit annotations for each failed check, plus upload the JSON/JUnit as artifacts.
Contributor guide
Research direction
Start with scripts/Invoke-UnitTests.ps1 and trace how per-module results, $moduleResults, exit codes, logs, and console output are produced; run the existing unit-test invocation to establish its current behavior. Done means optional JSON and JUnit outputs contain overall, module, and per-check results while existing outputs remain unchanged, with README or header documentation covering the new paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, powershell
- Domain
- ci-cd, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100