Automate Playwright testing during CI
- Dominant language
- JavaScript
- Stars
- 15.7k
- Forks
- 3.9k
- Avg merge
- 4d 6h
- Merged PRs (30d)
- 34
Description
One of the slowest parts of our current release process is running our end to end tests through Playwright. It's also very late in the process to catch any serious errors that could prevent or hold up doing the release. Playwright has [fairly robust docs](https://playwright.dev/docs/ci) for how to use it in CI but we currently don't so I'd like to start the discussion around why and whether we can enable it.
1. Why didn't we set it up with CI in the first place?
- Looking at the [initial implementation PR](https://github.com/CesiumGS/cesium/pull/11348) it just wasn't priority at that time. I'd argue it maybe should be at this point.
2. Why are we not including snapshots in the repo itself? Is it a repo space issue?
- I suspect one reason for this is the difference in rendering between systems. If that's the case I would still assume the _large_ majority of our tests _will_ look identical on all systems and we should set up a system to mark the ones that don't.
3. Why are we only running the e2e tests on releases? Is it just because they're too slow to run every pr? (CI might help reduce the need to run them locally)
This issue also closely couples with https://github.com/CesiumGS/cesium/issues/12281 because we will need to be much more precise with certain tests (like particles) so they actually _are_ perfectly reproducible every time.
Contributor guide
Research direction
Start with the Playwright CI documentation and the initial implementation PR (#11348), then review how the current end-to-end tests run during releases. Consider the reproducibility work in issue #12281 before defining the CI approach. Done should include an agreed, repeatable way to run the Playwright tests earlier in the workflow, with snapshot handling addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, playwright
- Domain
- ci-cd, release, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100