ampproject / ampproject/amphtml
Tracking Issue: Flaky (Bento?) unit tests
- Dominant language
- JavaScript
- Stars
- 14.9k
- Forks
- 4.1k
- PR merge metrics
- No merged PRs in 30d
Description
### Description
This is a tracking issue to organize information all in one place. We have recently seen an occasional unit test run fail with ~160 failing test cases. From what I can tell these are almost always Bento (1.0) components or their Preact counterparts.
Failures usually occur:
- for PRs making unrelated changes, indicating likelihood of flakiness rather than a code change causing these failures.
- on Circle CI (not locally)
- due to timeouts
[Example run](https://app.circleci.com/pipelines/github/ampproject/amphtml/11169/workflows/ad71e37c-5571-40f7-afd6-e7cf2e2e4cc6/jobs/176118)
### Reproduction Steps
There are no reliable reproduction steps, but #34630 was referenced as one where unrelated unit test failures of this nature happened after multiple re-runs of the unit task. @rileyajones has recommended debugging via VSCode which will be the next step here.
### Relevant Logs
_No response_
### Browser(s) Affected
_No response_
### OS(s) Affected
_No response_
### Device(s) Affected
_No response_
### AMP Version Affected
_No response_
Contributor guide
Research direction
Start with the example CircleCI run and the unit task referenced in #34630, then follow the recommended VSCode debugging path for the Bento 1.0 and Preact failures. Done means the intermittent timeout cause is identified and the affected unit tests no longer fail on unrelated PRs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100