ampproject / ampproject/amphtml

Tracking Issue: Flaky (Bento?) unit tests

Open
#34,885 1 comment 0 reactions 0 assignees View on GitHub
Stale Type: Bug WG: bento
Dominant language
JavaScript
Stars
14.9k
Forks
4.1k
PR merge metrics
No merged PRs in 30d

Description

### Description

This is a tracking issue to organize information all in one place. We have recently seen an occasional unit test run fail with ~160 failing test cases. From what I can tell these are almost always Bento (1.0) components or their Preact counterparts.

Failures usually occur:
- for PRs making unrelated changes, indicating likelihood of flakiness rather than a code change causing these failures.
- on Circle CI (not locally)
- due to timeouts

[Example run](https://app.circleci.com/pipelines/github/ampproject/amphtml/11169/workflows/ad71e37c-5571-40f7-afd6-e7cf2e2e4cc6/jobs/176118)

### Reproduction Steps

There are no reliable reproduction steps, but #34630 was referenced as one where unrelated unit test failures of this nature happened after multiple re-runs of the unit task. @rileyajones has recommended debugging via VSCode which will be the next step here.

### Relevant Logs

_No response_

### Browser(s) Affected

_No response_

### OS(s) Affected

_No response_

### Device(s) Affected

_No response_

### AMP Version Affected

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the example CircleCI run and the unit task referenced in #34630, then follow the recommended VSCode debugging path for the Bento 1.0 and Preact failures. Done means the intermittent timeout cause is identified and the affected unit tests no longer fail on unrelated PRs.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.