wp-media / wp-media/wp-rocket-e2e
Flaky test round 3
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1
- Forks
- 1
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 3
Description
Shared reports with AI:
scp auto-e2e-wpr-apache@57.128.123.120:~/wp-rocket-e2e/test-results-storage/2026-09-18T06-19-27-487Z/cucumber-report.html
scp auto-e2e-wpr@57.128.123.120:~/wp-rocket-e2e/test-results-storage/2026-09-18T06-06-37-576Z/cucumber-report.html
scp auto-e2e-wpr@57.128.123.120:~/wp-rocket-e2e/test-results-storage/2026-09-17T17-37-29-234Z/cucumber-report.html
scp auto-e2e-wpr@57.128.123.120:~/wp-rocket-e2e/test-results-storage/2026-09-17T11-07-14-269Z/cucumber-report.html
scp auto-e2e-wpr-apache@57.128.123.120:~/wp-rocket-e2e/test-results-storage/2026-09-17T06-24-18-962Z/cucumber-report.html
scp auto-e2e-wpr@57.128.123.120:~/wp-rocket-e2e/test-results-storage/2026-09-16T19-44-53-375Z/cucumber-report.html
scp auto-e2e-wpr@57.128.123.120:~/wp-rocket-e2e/test-results-storage/2026-09-16T14-58-20-752Z/cucumber-report.html
scp auto-e2e-wpr-apache@57.128.123.120:~/wp-rocket-e2e/test-results-storage/2026-09-16T11-23-50-714Z/cucumber-report.html
scp auto-e2e-wpr@57.128.123.120:~/wp-rocket-e2e/test-results-storage/2026-09-16T05-01-42-254Z/cucumber-report.html
Findings:
Cross-cutting root causes (affect many features)
-
SSH connectivity to the target test site (146.59.243.71) — pure infra, not test-code flakiness
Error: connect ECONNREFUSED 146.59.243.71:22 (12×) and Error: read ECONNRESET (9×) hit inside [hook]s across lcp-beacon-script.feature, the whole ll-css-bg-image* family, and others. This is the SSH link used to read/verify server-side state on the WP test site — separate from the 57.128.123.120 runner box. It looks like the SSH daemon on the target site drops/refuses connections under load, likely from the same CPU/resource contention already flagged in [[project_wp_rocket_e2e_activate_plugin_contention]]. Improvement: add SSH retry/backoff around that call instead of failing the whole hook on first refusal, and check whether the target site's sshd has connection-rate limits that concurrent suites are tripping. -
Fixed 5s/30s waits vs. a loaded shared box — the single biggest cluster
page.waitForLoadState('networkidle') / waitForSelector('a:has-text("Activate Plugin")') timing out at 30s on the "plugin is activated" step — matches the leading hypothesis already in memory ([[project_wp_rocket_e2e_activate_plugin_contention]]): concurrent auto-e2e suites (wpr, wpr-apache, bkwp, bkwp-apache) contending for CPU on one shared OVH box.
Same shape on "I am logged in" (page.waitForURL timeout) — recurs in delay-js, lcp-beacon-script, settings-export-import, notices/renewal-notice, notices/cpcss-notice, ll-css-bg-image, imagify. Login redirect is slow, not broken.
Same shape on "I must not see any error in debug.log" and "I enable all settings" — both use expect(...).toBeHidden() with a 5s timeout waiting for a loading overlay/spinner to disappear (37 + 5 occurrences, by far the largest single bucket, all in delay-js.feature and delete-plugin.feature/enable-all-features.feature).
Improvement: these all fail the same way (fixed short timeout on a box under variable load) → raise timeouts on load-dependent waits (or make them adaptive/poll-based) rather than adding retries as a band-aid. This is consistent with [[feedback_root_cause_fixes]] — fix the wait condition, don't just widen the retry net.
Feature-specific issues (real bugs, not just timing)
3. Mobile-menu step has state leakage — Error: Mobile menu is already open before attempting to open it in delay-js.feature's "expand mobile menu" scenarios. The step assumes the menu starts closed but doesn't verify/reset state first. Fix: have the step check current state and close-then-open, instead of assuming closed.
-
settings-export-import.feature: "Nothing changed in settings '3' vs '2'" (expect(received).toBeTruthy(), always false) — 8/15 attempts fail. Genuinely flaky (mixes pass/fail across runs), pointing to a race between the settings save and the comparison read — the comparison step likely runs before the export/save has fully persisted. Fix: wait on a concrete save-confirmation signal instead of a fixed delay.
-
imagify.feature: "Cannot save setting 'lazyload': section 'media' does not exist on the current page" (9× total, across lazyload/lazyloadYoutube/lazyloadIframes). The settings page section isn't rendered yet when the step tries to interact — a load-state race in general.ts:76, not random flake. Fix: wait for the section element itself, not just page load.
-
settings-export-import.feature: strict-mode locator violation — getByText('Database optimization process is') matches two
elements (running vs. complete) because the text is a prefix of both. Real selector bug, not flaky — fix: match the full string or use a more specific locator.
-
performance-hints.feature: "Required test page 'atf-lrc-1' does not exist. Template loader plugin may have failed." (3×) — environment/fixture provisioning failure on the target site, not a script bug. Worth checking whether this correlates with the same box-contention window as #1/#2.
Bottom line
Roughly 70% of the failures trace back to two systemic causes already suspected in memory: (a) shared-box CPU contention making fixed-timeout waits ("plugin is activated", "I am logged in", debug.log check) flaky, and (b) SSH flakiness to the target site under the same load. The remaining ~30% are three concrete, fixable step-definition bugs (#3 mobile-menu state, #5 imagify section-wait, #6 strict-mode locator) plus one real race condition (#4 export/import compare) — those are worth fixing directly rather than retrying around, per your root-cause-first preference.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the shared cucumber reports and the failures in delay-js.feature, settings-export-import.feature, and performance-hints.feature. Inspect the related step definitions, including general.ts:76, then reproduce the failures under the reported shared-box conditions. Done means the identified timeout, state, persistence, locator, and fixture failures are addressed and the affected end-to-end scenarios pass reliably.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- playwright, typescript
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100