Skipped unit tests should not show as "passed"
- Dominant language
- Dockerfile
- Stars
- 4.9k
- Forks
- 2k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 26
Description
We have lots of instances of skipping tests in this manner:
https://github.com/dotnet/dotnet-docker/blob/ccbb4f2e87307ef063c576f638ee2416d7a5d57e/tests/Microsoft.DotNet.Docker.Tests/RuntimeDepsImageTests.cs#L33-L38
However, returning early from a test will count as a "passed" test in Xunit, even if we didn't assert anything. This can lead to a false sense of confidence in test outcomes, especially for tests which are intended to be skipped temporarily.
We should find a way to make these tests show as "skipped" instead, like we do with Theories that have no data due to test filtering.
Xunit V3 (which is in preview) supports dynamic skipping of tests natively, but not Xunit V2. There's opportunity to use a library like [Xunit.SkippableFact](https://www.nuget.org/packages/Xunit.SkippableFact), which is used by several other Microsoft projects: [Xunit.SkippableFact > used by ...](https://www.nuget.org/packages/Xunit.SkippableFact#usedby-body-tab).
Contributor guide
Assessment
This issue has not been assessed yet.