shopware / shopware/shopware

Add playwright screenshot tests where it makes sense

Open
#10,637 0 comments 0 reactions 0 assignees View on GitHub
domain/crm-after-sales extension/Migration-Assistant priority/low
Dominant language
PHP
Stars
3.4k
Forks
1.2k
Avg merge
3d 55m
Merged PRs (30d)
436

Description

Add some screenshot tests to the existing playwright tests to ensure the UI isn’t changing unexpectedly in a visual way.

It’s important that these tests are reliable and not flaky, so any data that could change without affecting the migration should be hidden from the screenshot. See [https://playwright.dev/docs/test-snapshots|https://playwright.dev/docs/test-snapshots|smart-link] and add a CSS file which hides all this unrelated data.

Example start of implementation: [https://gitlab.shopware.com/shopware/6/services/migration-assistant/-/merge_requests/475/diffs?commit_id=ea3c6fd317962d4a280f969fd5cf811961080ed9|https://gitlab.shopware.com/shopware/6/services/migration-assistant/-/merge_requests/475/diffs?commit_id=ea3c6fd317962d4a280f969fd5cf811961080ed9]

Reasons it was discard for the moment:

* we don't use it in either platform nor commercial yet and the playwright API is relatively new (style overrides were only introduced a few versions ago)
* the screenshots need to be commited and updated in case of differences, but local execution may cause differences on it's own (like operating system / different browser / ...)
* It's hard to not introduce flakiness, even with custom css to hide many things

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing Playwright tests, the Playwright test snapshots guidance, and the linked migration-assistant merge request. Identify suitable stable UI states and add the CSS file that hides unrelated changing data. Done means the selected screenshots are committed and repeatable without unrelated visual differences.

Written by the indexing model from the issue text.

Assessment

Tech stack
css
Domain
frontend, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.