Tracking parallel testing outstanding issues
- Dominant language
- F#
- Stars
- 4.3k
- Forks
- 876
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 131
Description
Parallel testing is now merged and gets a lot more runtime in the CI. This means some test cases that are not thread safe and were missed before may fail occasionally.
I think it would be good to have a central place to keep track of tests that we suspect are flaky because of concurrent execution. Other annoyances like testhost crashes, both locally and in the Azure pipeline, or any weird behavior are also worth reporting.
I noticed it is often more likely for a test to fail in CI and seem to pass locally. To reproduce locally I just select the whole module containing the test case and do "Run until failure" in VS Test Explorer.
Some issues noticed so far:
- [x] net9.0 CLR crashing with `Internal CLR error. (0x80131506)`. This is external and reported. (no longer happening with runtime 9.0.3)
- [x] `ProjectAnalysisTests.Test Project25 whole project errors` and other test cases using `Project25` are flaky. (Because of type provider?)
- [x] `Test project1 and make sure TcImports gets cleaned up` sometimes waits forever for GC.
- [x] tests executing `ildasm` sometimes [crash on mac](https://dev.azure.com/dnceng-public/public/_build/results?buildId=906033&view=logs&j=7bab896a-24f8-544f-51eb-43745367a332&t=999dbed9-85e3-59ab-57f0-3e22828b5bad&l=3545) (no longer happening with runtime 9.0.3)
Contributor guide
Assessment
This issue has not been assessed yet.