WordPress / WordPress/block-development-examples

E2E suite silently tests nothing — revive it and run it in CI

Open
#147 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
334
Forks
52
PR merge metrics
No merged PRs in 30d

Description

Problem

_tests/e2e/insertion.spec.js derives the plugins to test from the plugins array in .wp-env.json. The local ./plugins/... entries were removed in the repo revamp (f0b9107), leaving only the Gutenberg zip — so pluginsPathsToTest is empty and npm run test:e2e passes with zero tests. E2E is also not wired into any GitHub workflow, so nothing could have caught this: any PR can break a block's build or registration and merge green.

Related rot discovered while auditing:

  • All 23 files in _tests/e2e/__snapshots__/ are orphaned — the current spec uses toContain(), not snapshots. One references a plugin that no longer exists (...833d15...).
  • The spec imports @wordpress/e2e-test-utils-playwright, which is not a root devDependency; it resolves only transitively through @wordpress/scripts, which is fragile under pnpm (the lockfile contains two different versions of it).
  • plugins/editable-block-1b8c51/e2e/basic.spec.js uses the deprecated Puppeteer stack and contains assertions that cannot fail (expect( true ).toBe( true ); expect( page.$( ... ) ) on an unawaited Promise). Seven other plugins declare test:e2e/test:unit scripts with no test files.
Proposed fix
  1. Derive the plugin list from the filesystem so it can't rot again: keep .wp-env.json static and map the whole plugins/ directory, then activate all plugins via WP-CLI in a Playwright global-setup. The spec walks plugins/*/ directly instead of parsing .wp-env.json.
  2. Add a CI workflow that runs the build, starts wp-env, and runs test:e2e on every pull request and on push to trunk.
  3. Delete _tests/e2e/__snapshots__/.
  4. Add @wordpress/e2e-test-utils-playwright as an explicit root devDependency.
  5. Delete plugins/editable-block-1b8c51/e2e/ and the dead test:e2e/test:unit scripts from plugins without test files.
Out of scope
  • Behavioural coverage (Interactivity API front-end assertions, SlotFill/sidebar/format-api tests) — this issue only restores the insertion smoke-test safety net.
  • A purpose-built "e2e testing for blocks" teaching example (possible follow-up, replacing what editable-block-1b8c51/e2e/ gestured at).
Notes for implementers
  • wp-env only auto-activates plugins listed in its plugins array; a mappings mount does not activate them — hence the WP-CLI activation step.
  • Watch for inter-plugin interference with all 30 active at once (several register post-meta UIs and sidebars). If conflicts appear, activation may need to be per-test-group.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running npm run test:e2e and reading _tests/e2e/insertion.spec.js, .wp-env.json, and the package scripts to confirm why no tests run. Then inspect the existing workflow directory and the listed snapshot and editable-block-1b8c51/e2e files. Done means the insertion smoke test runs against the filesystem plugins, the suite is exercised on pull requests and trunk pushes, and the obsolete test assets and scripts are removed.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, javascript, playwright
Domain
ci-cd, devops, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.