Azure / Azure/azure-sdk-tools

Add CI gates on script changes and missing tests

Open
#3,472 0 comments 0 reactions 0 assignees View on GitHub
Central-EngSys
Dominant language
C#
Stars
135
Forks
260
Avg merge
3d 1h
Merged PRs (30d)
144

Description

We need to start getting better at unit testing our scripts, and structuring the scripts to be more modular and easily testable.

We already have a pipeline template for pester tests: https://github.com/Azure/azure-sdk-tools/blob/main/eng/pipelines/templates/stages/archetype-sdk-tool-pwsh.yml and examples of test suites: [here](https://github.com/Azure/azure-sdk-tools/blob/main/tools/asset-sync/assets.Tests.ps1) and [here](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/scripts/job-matrix/tests/job-matrix-functions.tests.ps1).

I think a good start should be to soft require tests in pull request reviews, with the eventual goal of running the above pipeline for all script changes, especially those in `eng/common`. Additionally, we should limit the use of inline scripts in devops yaml, as it is not testable, by disallowing it in review, or perhaps requiring some sort of linting+override.

We could also consider higher level language agnostic integration tests that could be used to test programs getting refactored out of powershell and into another language, if we move in that direction.

Contributor guide

Open the contributing guide

Research direction

Start by reading eng/pipelines/templates/stages/archetype-sdk-tool-pwsh.yml and the example suites in tools/asset-sync/assets.Tests.ps1 and eng/common/scripts/job-matrix/tests/job-matrix-functions.tests.ps1. The issue does not define a specific gate or acceptance criteria; done would require an agreed scope for soft-required tests, script-change coverage, and inline DevOps YAML handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
powershell
Domain
ci-cd, devops, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.