ampproject / ampproject/amphtml

Gate new and updated tests based on performance and flakiness

Open
#35,443 2 comments 0 reactions 0 assignees View on GitHub
INTENT TO DEPRECATE Stale
Dominant language
JavaScript
Stars
14.9k
Forks
4.1k
PR merge metrics
No merged PRs in 30d

Description

### Summary

Check newly created and updated tests meet performance and flakiness thresholds and enable developers to meet these standards.
I wrote a this design doc on [Keeping AMP Tests Fast](https://docs.google.com/document/d/1o9AaJ8-eKMRxcrLrqLQwF5ZUJeV8AELu4uT8mdec9wI) describing how gating might work.

### Motivation

There is a significant number of slow running tests that have been introduced either unintentionally through bugs in test code or else by being unaware of the ramifications of test delays. An effort is underway to resolve the existing issues, but something should be done to prevent the issues from arising again in the future.

### Impact on Existing Users

Users may have to deal with increased friction when added and updating tests.

### Alternative Implementation

Periodically perform manual audits and attempt to improve tests which perform poorly.

### Additional Context

_No response_

### Notifications

/cc @ampproject/wg-approvers

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked “Keeping AMP Tests Fast” design document and reviewing how AMP tests are currently run and checked. Done means newly created and updated tests are gated against performance and flakiness thresholds, with a documented path for developers to meet those standards.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
ci-cd, performance, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.