Automattic / Automattic/blocks-engine
Run solved fixture regression gate in GitHub Actions
- Dominant language
- PHP
- Stars
- 14
- Forks
- 2
- Avg merge
- 2h 10m
- Merged PRs (30d)
- 561
Description
## Problem
PHP Transformer changes can regress fixtures that have already graduated into `fixtures/solved/`, but Blocks Engine currently has no required GitHub check that executes those permanent fixtures through the real Static Site Importer and WordPress runtime.
The owning SSI lane is tracked in Automattic/static-site-importer#668.
## Contract
- Add a GitHub Actions workflow owned by Blocks Engine.
- Trigger it for relevant PHP Transformer, shared transformer, fixture-contract, and workflow changes, with manual dispatch support.
- Check out a pinned, reviewed Static Site Importer harness revision.
- Run SSI’s solved-corpus-only lane through `homeboy bench` using the current Blocks Engine checkout as the PHP Transformer override.
- Execute real import, editor validation, native-editability, exact visual-parity, and unacceptable-finding gates through WP Codebox.
- Upload replayable Homeboy summaries and WP Codebox evidence, including screenshots/diffs on failure.
- Fail when the solved corpus is empty or any fixture becomes `solved_regression`.
- Keep the workflow deterministic and use one Homeboy bench iteration; this is a quality gate, not a performance benchmark.
## Acceptance
1. The workflow tests the PR checkout rather than a released transformer.
2. A deliberate solved-fixture regression fails the check.
3. Current trunk passes or produces a directly actionable tracked baseline failure.
4. Artifacts identify the Blocks Engine commit, SSI harness commit, selected fixture count, and Homeboy run.
5. The check can be made required after a successful baseline run.
Depends on Automattic/static-site-importer#668.
## AI assistance
- **AI assistance:** Yes
- **Tool(s):** OpenCode with OpenAI GPT-5.6 Sol
- **Used for:** Investigated the existing Homeboy, SSI fixture-matrix, and WP Codebox contracts with Chris and drafted this cross-repo CI tracker.
Contributor guide
Research direction
Start by reading the existing Homeboy, SSI fixture-matrix, and WP Codebox contracts, then run the solved-corpus-only lane through homeboy bench with the current Blocks Engine checkout. Done means the workflow is deterministic, fails on solved regressions or an empty corpus, and uploads artifacts identifying both commits, the fixture count, the run, and failure evidence; first resolve the dependency on Automattic/static-site-importer#668.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, php, wordpress
- Domain
- ci-cd, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100