erigontech / erigontech/erigon
ci: move flaky tests off main/release branch triggers
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 455
Description
Flaky tests currently run on `main` and `release/**` branches. Failures there can't be corrected — there's no PR to fix, no author to notify. The branch looks broken to outside observers.
Flaky tests should not run on main or release if they're known to be flaky. The preferred destination is `pull_request` triggers where failures are visible to the author and can be prioritised. However, some flaky tests are also too long-running for PR checks — this is an unsolved problem. Even so, running them on main/release is not the right answer since failures there are permanent and public.
For long-running flaky tests that can't move to PR checks, the failure attribution mechanism in #20300 would at least make scheduled failures actionable by linking them to the commit that introduced the regression.
Known flaky tests to evaluate:
- #19446 — Flaky Hive test "Blob Transaction Ordering"
- #15939 — Flaky TestState/stEIP1559 timeout
- #8789 — Unstable bellatrix ssz_static test
- #14975 — TestEvmRun failing on Windows CI
- #14468 — global variable race in tests
Related: #19136, #20300
Contributor guide
Assessment
This issue has not been assessed yet.