ampproject / ampproject/amphtml

Investigate running AMP tests on Safari Technology Preview using MacOS VMs

Open
#27,044 5 comments 0 reactions 0 assignees View on GitHub
P3: When Possible Stale Type: Feature Request WG: infra
Dominant language
JavaScript
Stars
14.9k
Forks
4.1k
PR merge metrics
No merged PRs in 30d

Description

We'd like to be able to run (a subset of) AMP tests on [Safari Technology Preview](https://developer.apple.com/safari/technology-preview/). Some things to investigate:

- Compare Travis's [MacOS VMs](https://docs.travis-ci.com/user/reference/osx/#using-macos) against [Github Actions](https://help.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners). (#24088 tried this a few months ago.)
- We should aim to run unit, integration, and e2e tests. Do they run fine and pass?
- What is the start-up time of the VMs, and running time of the tests?
- Based on the above answers, should the tests be part of PR builds, push builds, or run when a new nightly branch is cut?

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the approach attempted in #24088 and the Travis macOS VM and GitHub Actions virtual-environment documentation linked in the issue. Run the unit, integration, and end-to-end AMP test subsets on Safari Technology Preview, record VM startup and test durations, and document whether they should run on PR builds, push builds, or nightly branches.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, javascript, macos
Domain
ci-cd, devops, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.