`OuterLoop`, and `SkipOnPlatform` attributes conflict, and don't work correctly when used together
- Dominant language
- C#
- Stars
- 189
- Forks
- 67
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 29
Description
Consider
https://github.com/dotnet/runtime/blob/978df67ced885aeca5f7e75379451bc1a57cc219/src/libraries/System.Net.WebSockets/tests/WebSocketCreateTest.cs#L40-L45
```csharp
[OuterLoop("Uses external servers")]
[Theory]
[MemberData(nameof(EchoServers))]
[SkipOnPlatform(TestPlatforms.Browser, "System.Net.Sockets is not supported on this platform.")]
[ActiveIssue("https://github.com/dotnet/runtime/issues/34690", TestPlatforms.Windows, TargetFrameworkMonikers.Netcoreapp, TestRuntimes.Mono)]
public async Task WebSocketProtocol_CreateFromConnectedStream_CanSendReceiveData(Uri echoUri)
```
This has both `OuterLoop`, and `SkipOnPlatform` attributes. This should run when using `category=outerloop`, but get skipped if it is running on the browser. Instead, the tests run anyway, and fail because the operations are not supported on browser.
I can reproduce this with `./dotnet.sh build /bl src/libraries/System.Net.WebSockets/tests/System.Net.WebSockets.Tests.csproj /p:Configuration=Release /p:TargetOS=Browser /p:TargetArchitecture=wasm /t:Test -p:Scenario=WasmTestOnBrowser -p:TestScope=outerloop -p:XUnitClassName=System.Net.WebSockets.Tests.WebSocketTests`
this is the xharness command line:
` wasm test-browser --app=. --output-directory=/workspaces/runtime/artifacts/bin/System.Net.WebSockets.Tests/Release/net7.0/browser-wasm/AppBundle/xharness-output
--browser-arg=--no-sandbox -s dotnet.js.symbols --symbol-patterns wasm-symbol-patterns.txt --browser-arg=--no-sandbox -- --run WasmTestRunner.dll System.Net.WebSockets.Tests.dll -class System.Net.WebSockets.Tests.WebSocketTests -trait category=OuterLoop -notrait category=failing`.
I don't think this is wasm, or `SkipOnPlatform` specific.
cc @akoeplinger @premun
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.