Add 6.8 major-feature visual regression coverage for Administration
- Dominant language
- PHP
- Stars
- 3.4k
- Forks
- 1.2k
- Avg merge
- 3d 55m
- Merged PRs (30d)
- 436
Description
### User story
## Context
The current visual regression suite runs Playwright tests tagged `@Visual` in the `Visual` project. It exercises the default feature configuration and compares screenshots against committed `*-Visual-linux.png` baselines.
The 6.8 Administration path is activated through `FEATURE_ALL=major` and can render different components and layouts, including `mt-tabs`. Its screenshots must not reuse the default-path baselines: some differences are intentional, while unintended differences need review.
This is the automated follow-up to the manual audit of `mt-tabs` surfaces with `V6_8_0_0` enabled: https://github.com/shopware/shopware/issues/19903
## Goal
Run Administration visual regression tests with `FEATURE_ALL=major` and maintain a separate, reviewable screenshot baseline for the 6.8 path.
## Implementation (TBD)
1. Add a dedicated Playwright project, for example `VisualMajor`, based on the existing `Visual` project:
- Chromium
- `@Visual` grep
- same deterministic viewport and setup
- distinct snapshot suffix/baseline, e.g. `*-VisualMajor-linux.png`
2. Extend the visual-test workflow with a major-feature lane:
- pass `major: true` to the ATS composite action so it exports `FEATURE_ALL=major` before Shopware is installed and built;
- run the `VisualMajor` project with one worker;
- upload its traces, report, and screenshot differences separately from the default `Visual` lane.
3. Create and commit the initial `VisualMajor` baseline snapshots.
4. Ensure the visual suite covers the 6.8-specific `mt-tabs` paths identified by the audit. Add deterministic visual scenarios where current `@Visual` coverage does not reach them, at least:
- Product media modal
- Media folder settings modal
- Flow Builder listing
- Settings → Search → Searchable content
### Acceptance criteria
- A CI workflow can run the complete visual suite with `FEATURE_ALL=major`.
- The 6.8 run uses separate snapshot files from the default feature path.
- Screenshot results and traces are available for review for both lanes.
- Initial baselines are reviewed and committed.
- The identified `mt-tabs` surfaces are covered by deterministic visual tests.
- Existing default-path visual regression testing remains unchanged.
### Definition of Done
- [ ] Fulfills all acceptance criteria defined during discovery.
- [ ] Integration/E2E testing in staging is done.
- [ ] All integration/E2E/unit tests passing; all critical or high-priority bugs are resolved.
- [ ] Fulfills compliance, performance, security, and cloud-readiness needs.
- [ ] Observability—includes monitoring, alerting, and logging; incident response handbooks updated.
- [ ] Documentation—developer docs are written or updated, including information on how the feature or change is adopted and tested. If new functionality, end user documentation is written or updated.
- [ ] (conditional) All new API endpoints have a defined API schema.
- [ ] (conditional) If the change affects the Storefront, it has been tested in the latest three versions of Firefox, Chrome, Edge, IE11, and Safari, as well as the different viewports and devices (Desktop / Tablet / Mobile).
- [ ] (conditional) If the change affects the Administration, it has been tested in current versions of Firefox, Chrome, and Edge.
- [ ] (conditional) Functionality that includes new API routes are callable via the Store-API (/store-api/[version]/).
- [ ] (conditional) If a new Admin module is created, the access control layer is provided, and permissions can be managed via the Users & Permissions module.
- [ ] (conditional) If a new UI input field or interactive element is introduced, the ACL wrapper is added.
- [ ] (conditional) Performance testing done if the change has a high performance impact, such as affecting scalability, load balancing, or query optimization.
Contributor guide
Research direction
Start with the existing Playwright Visual project and @Visual tests, then inspect the visual-test workflow and ATS composite action to trace how configuration and snapshots are handled. Add the VisualMajor lane with FEATURE_ALL=major, separate baseline and artifacts, and deterministic coverage for the Product media modal, Media folder settings modal, Flow Builder listing, and Settings → Search → Searchable content. Done means reviewed initial baselines, separate CI results and traces, and unchanged default-path coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- playwright
- Domain
- ci-cd, frontend, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100