Add CI gates on script changes and missing tests
- Dominant language
- C#
- Stars
- 135
- Forks
- 260
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 144
Description
We need to start getting better at unit testing our scripts, and structuring the scripts to be more modular and easily testable.
We already have a pipeline template for pester tests: https://github.com/Azure/azure-sdk-tools/blob/main/eng/pipelines/templates/stages/archetype-sdk-tool-pwsh.yml and examples of test suites: [here](https://github.com/Azure/azure-sdk-tools/blob/main/tools/asset-sync/assets.Tests.ps1) and [here](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/scripts/job-matrix/tests/job-matrix-functions.tests.ps1).
I think a good start should be to soft require tests in pull request reviews, with the eventual goal of running the above pipeline for all script changes, especially those in `eng/common`. Additionally, we should limit the use of inline scripts in devops yaml, as it is not testable, by disallowing it in review, or perhaps requiring some sort of linting+override.
We could also consider higher level language agnostic integration tests that could be used to test programs getting refactored out of powershell and into another language, if we move in that direction.
Contributor guide
Research direction
Start by reading eng/pipelines/templates/stages/archetype-sdk-tool-pwsh.yml and the example suites in tools/asset-sync/assets.Tests.ps1 and eng/common/scripts/job-matrix/tests/job-matrix-functions.tests.ps1. The issue does not define a specific gate or acceptance criteria; done would require an agreed scope for soft-required tests, script-change coverage, and inline DevOps YAML handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- powershell
- Domain
- ci-cd, devops, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100