[ci] Test retries don't clean between runs
- Dominant language
- C#
- Stars
- 2.1k
- Forks
- 579
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 257
Description
For the tests that use Azure Pipelines' built-in `retryCountOnTaskFailure` (like `Mono.Android.NET-Tests`), the test suite is re-run immediately after the failure. We cannot inject any extra steps before the retry. This means that the second run is an incremental build because the `obj`/`bin` directories are not cleaned. Additionally, the logs from the first run are not captured.
This could lead to passing of tests that fail on initial builds and only succeed on incremental builds.
Given the inflexibility of `retryCountOnTaskFailure`, we likely need to turn it off completely.
We should also examine our usage of `dotnet-test-slicer` to see if it suffers from the same issue. At least with `dotnet-test-slicer` we can run steps between the initial run and the retry, so we can add manual cleans if needed.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.