CesiumGS / CesiumGS/cesium

Automate Playwright testing during CI

Open
#12,355 2 comments 0 reactions 0 assignees View on GitHub
category - packaging & tooling theme - developer comfort
Dominant language
JavaScript
Stars
15.7k
Forks
3.9k
Avg merge
4d 6h
Merged PRs (30d)
34

Description

One of the slowest parts of our current release process is running our end to end tests through Playwright. It's also very late in the process to catch any serious errors that could prevent or hold up doing the release. Playwright has [fairly robust docs](https://playwright.dev/docs/ci) for how to use it in CI but we currently don't so I'd like to start the discussion around why and whether we can enable it.

1. Why didn't we set it up with CI in the first place?
- Looking at the [initial implementation PR](https://github.com/CesiumGS/cesium/pull/11348) it just wasn't priority at that time. I'd argue it maybe should be at this point.
2. Why are we not including snapshots in the repo itself? Is it a repo space issue?
- I suspect one reason for this is the difference in rendering between systems. If that's the case I would still assume the _large_ majority of our tests _will_ look identical on all systems and we should set up a system to mark the ones that don't.
3. Why are we only running the e2e tests on releases? Is it just because they're too slow to run every pr? (CI might help reduce the need to run them locally)

This issue also closely couples with https://github.com/CesiumGS/cesium/issues/12281 because we will need to be much more precise with certain tests (like particles) so they actually _are_ perfectly reproducible every time.

Contributor guide

Open the contributing guide

Research direction

Start with the Playwright CI documentation and the initial implementation PR (#11348), then review how the current end-to-end tests run during releases. Consider the reproducibility work in issue #12281 before defining the CI approach. Done should include an agreed, repeatable way to run the Playwright tests earlier in the workflow, with snapshot handling addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, playwright
Domain
ci-cd, release, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.