Azure / Azure/azure-cli

Exclude certain tests when running azdev test

Open
#16,204 1 comment 0 reactions 1 assignee Assigned to @kairu-ms View on GitHub
feature-request Test Framework
Dominant language
Python
Stars
4.6k
Forks
3.5k
Avg merge
3d 2h
Merged PRs (30d)
60

Description

**Is your feature request related to a problem? Please describe.**
It is not really a problem, but it would be of great convenience. In our module, 'rdbms', we have quite a large number of tests that take around ~3-4 hours just for flexible servers. We run our tests in Stage whenever a backend train is deployed. However, vnets are not supported in our stage region. As a result, we always have to comment out the vnet tests in our module or run the individual test classes like azdev test , azdev test , etc. when we run the tests in Stage. We are planning to automate this in a pipeline. However, in our pipeline, we cannot comment out tests as they are directly pulling from the master branch(azure-cli) and running the tests in the build agent. As a result, even though we know that the vnet tests would fail, we have to wait for all the tests to complete . We have around 10 tests in vnet whose run is unnecessary when we are running the pipeline in Stage.

**Describe the solution you'd like**
I am looking for something like
azdev test test_rdbms_flexible_commands --live --exclude < List of TestClassNames | List of TestFunctionNames>

**Describe alternatives you've considered**
1. Manually Commenting out tests that I dont want to run everytime
2. Individually run the test classes in the file except the tests I dont want to run.

**Additional context**
We have a automated pipeline that we run regularly in our Stage environment. We just have to ignore the failures currently as there is no way I can exclude the tests to run when I automated the pipeline. I wish I had something like azdev test test_rdbms_flexible_commands --live --exclude FlexibleServerVnetMgmtScenarioTest so that I can get to see see the complete scneario for our test results

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.