dotnet / dotnet/BenchmarkDotNet
Add settings to skip tests that take times on draft PR
- Dominant language
- C#
- Stars
- 11.5k
- Forks
- 1.1k
- Avg merge
- 6d 11h
- Merged PRs (30d)
- 12
Description
Currently, BenchmarkDotNet CI tests takes about 1.5h - 3h. (Especially slow on `macos(x64)` image)
And it cause following issues.
1. CI tests turnaround time is too long if there are failed tests.
2. It takes GitHub Actions compute resources
(e.g. Running CI on `macos` 3h takes about $11, it's not charged on OSS project though)
So, It's needed to reduce CI execution times on PR.
---
**Expected workaround**
Add setting to run **full** tests after `Ready for Review`.
When draft PR is created, Run only limited set of tests.
1. Skip some tests that are take long times (It's not works inProcessToolchain, and require build/publish operations)
2. Skip Wasm/NativeAOT/R2R related tests and installation of `v8`/`wasm-tool` workload
3. Skip CI on `macos(x64)` image
Contributor guide
Assessment
This issue has not been assessed yet.