dotnet / dotnet/BenchmarkDotNet

Add settings to skip tests that take times on draft PR

Open
#3,088 0 comments 0 reactions 0 assignees View on GitHub
Area:DevOps
Dominant language
C#
Stars
11.5k
Forks
1.1k
Avg merge
6d 11h
Merged PRs (30d)
12

Description

Currently, BenchmarkDotNet CI tests takes about 1.5h - 3h. (Especially slow on `macos(x64)` image)
And it cause following issues.

1. CI tests turnaround time is too long if there are failed tests.
2. It takes GitHub Actions compute resources
(e.g. Running CI on `macos` 3h takes about $11, it's not charged on OSS project though)

So, It's needed to reduce CI execution times on PR.

---

**Expected workaround**
Add setting to run **full** tests after `Ready for Review`.

When draft PR is created, Run only limited set of tests.
1. Skip some tests that are take long times (It's not works inProcessToolchain, and require build/publish operations)
2. Skip Wasm/NativeAOT/R2R related tests and installation of `v8`/`wasm-tool` workload
3. Skip CI on `macos(x64)` image

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.