Flaky CI: MTP/TRX host fails when suite filters select zero tests
- Dominant language
- C#
- Stars
- 10.9k
- Forks
- 2.1k
- Avg merge
- 13h 56m
- Merged PRs (30d)
- 351
Description
## Description
`Orleans.Analyzers.Tests` intermittently causes a filtered Microsoft.Testing.Platform CI job to fail when the active suite filter selects zero tests from that assembly. The suite continues and reports no failed tests, but the assembly exits unsuccessfully during test-host shutdown.
This has reproduced on unrelated pull requests and different operating systems/frameworks.
## Occurrences
### PR #10746
- Run/job: https://github.com/dotnet/orleans/actions/runs/32499721406/job/96826364553
- Environment: Functional, Windows, `net8.0`
- Head: `d1fe65475179923ba00ba6941fc077305151851a`
- PR scope: documentation only
```text
Orleans.Analyzers.Tests.dll (net8.0|x64) Zero tests ran (28s 485ms)
Exit code: 7
Standard output: Waiting 10 seconds for foreground threads to exit...
Error output: [FATAL ERROR] Foreground threads were left running, forcing process exit
```
The overall run reported 501 total tests, 482 succeeded, 19 skipped, 0 failed, and 1 test-application error.
### PR #10739
- Run: https://github.com/dotnet/orleans/actions/runs/32497565314
- Environment: SlowBVT, Ubuntu, `net10.0`
- PR scope: journaling recovery behavior and tests
```text
Orleans.Analyzers.Tests.dll (net10.0|x64) Zero tests ran (37s 377ms)
Exit code: 134
Unhandled exception. System.TimeoutException: The operation has timed out.
at Microsoft.Testing.Platform.Helpers.TaskExtensions.TimeoutAfterAsync(...)
at Microsoft.Testing.Extensions.TrxReport.Abstractions.TrxProcessLifetimeHandler...
```
## Expected behavior
A test application containing no tests which match the suite filter exits successfully and releases all MTP/TRX resources without delaying or failing the aggregate CI job.
## Notes
The failures occur in the analyzer test assembly under suite filters which select no analyzer tests. The different shutdown symptoms may share test-host or TRX lifetime cleanup behavior. The failed PR #10746 job is being rerun to confirm the intermittent nature.
Contributor guide
Research direction
Start with the Orleans.Analyzers.Tests test application and reproduce a filtered run that selects zero tests on Microsoft.Testing.Platform with TRX enabled. Trace the test-host shutdown and TRX lifetime cleanup, comparing the Windows net8.0 and Ubuntu net10.0 failures described here. Done means a zero-test filtered application exits successfully, releases its resources, and does not fail or delay the aggregate CI job.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100