Azure / Azure/azure-cli-dev-tools
Should run full test if there is API, SDK or CLI version bump
- Dominant language
- Python
- Stars
- 91
- Forks
- 134
- Avg merge
- 2h 51m
- Merged PRs (30d)
- 1
Description
Currently, for SDK or API version bump, only incremental tests are run. This frequently leads to `dev` branch Batched CI failure because the new API or SDK conflicts with other modules, mainly due to API version mismatch, such as https://github.com/Azure/azure-cli/pull/21681#issuecomment-1081334491:
https://dev.azure.com/azure-sdk/public/_build/results?buildId=1466412&view=logs&j=ad51cf76-b294-5bde-777e-be77cc5f3050&t=0e02ab23-4098-514d-f7b9-ab422251bbb2
```
2022-03-29T02:16:29.3378803Z except CannotOverwriteExistingCassetteException as ex:
2022-03-29T02:16:29.3379238Z > raise AssertionError(ex)
2022-03-29T02:16:29.3380350Z E AssertionError: Can't overwrite existing cassette ('/opt/az/lib/python3.8/site-packages/azure/cli/command_modules/monitor/tests/latest/recordings/test_metric_alert_for_sql_database_scope.yaml') in your current record mode ('once').
2022-03-29T02:16:29.3382083Z E No match for the request () was found.
2022-03-29T02:16:29.3383114Z E Found 1 similar requests with 1 different matcher(s) :
2022-03-29T02:16:29.3383519Z E
2022-03-29T02:16:29.3384713Z E 1 - ().
2022-03-29T02:16:29.3385863Z E Matchers succeeded : ['method', 'scheme', 'host', 'port', 'path']
2022-03-29T02:16:29.3386340Z E Matchers failed :
2022-03-29T02:16:29.3386940Z E _custom_request_query_matcher - assertion failure :
2022-03-29T02:16:29.3387367Z E None
2022-03-29T02:16:29.3387562Z
2022-03-29T02:16:29.3388186Z /opt/az/lib/python3.8/site-packages/azure/cli/testsdk/base.py:299: AssertionError
2022-03-29T02:16:29.3389118Z ----------------------------- Captured stdout call -----------------------------
2022-03-29T02:16:29.3389583Z None
2022-03-29T02:16:29.3390202Z ------------ generated xml file: /azure_cli_test_result/monitor.xml ------------
2022-03-29T02:16:29.3390794Z =========================== short test summary info ============================
2022-03-29T02:16:29.3391462Z FAILED tests/latest/test_monitor_metric_alert_scenarios.py::MonitorTests::test_metric_alert_for_sql_database_scope
```
The solution is to run full test if contents in these files are changed:
- `src/azure-cli-core/azure/cli/core/profiles/_shared.py`
- `src/azure-cli/requirements.py3.Darwin.txt`
- `src/azure-cli/requirements.py3.Linux.txt`
- `src/azure-cli/requirements.py3.windows.txt`
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.