ampproject / ampproject/amphtml
Non-deterministic flakiness in E2E tests
- Dominant language
- JavaScript
- Stars
- 14.9k
- Forks
- 4.1k
- PR merge metrics
- No merged PRs in 30d
Description
Of late, we're seeing an uptick in flakiness on our E2E tests. Many of the recent failures appear to be in the experiment tests run on the `master` branch. In addition, PR jobs are showing failures too.
[Here is a link](https://app.circleci.com/pipelines/github/ampproject/amphtml?branch=master) to the `master` builds on CircleCI. All failures in the past 24h seem to come from E2E tests. (**Pro-tip:** Enable "auto expand" if not already enabled, and refresh the page. You'll see every failing job on one screen without having to click through.)
This is a tracking bug to investigate and fix the root cause of the problem. (New Chrome version? A new dependency version that's causing more flakiness? Bit-rotting of E2E infrastructure code?)
Contributor guide
Assessment
This issue has not been assessed yet.