Automattic / Automattic/wp-codebox
Publish pinned release artifacts: consumers fall back to building @main every CI run, so every upstream regression breaks all downstream CI live
- Dominant language
- TypeScript
- Stars
- 16
- Forks
- 4
- Avg merge
- 59m
- Merged PRs (30d)
- 131
Description
## Summary
When the WP Codebox CLI release artifact is **not published**, downstream CI setups fall back to a **source install of `Automattic/wp-codebox@main` on every run**. That means every consumer's CI silently tracks tip-of-main, so **any regression merged to `wp-codebox@main` breaks all downstream CI immediately and simultaneously** — with no version pinning to insulate them. This turns wp-codebox `main` into an un-pinnable, always-latest dependency for every consumer's test gate.
## Concrete incident (Exhibit A)
On 2026-07-03, a mount-path change (PR #1698 "canonical-input-mount-paths", commit `242e36f8`) relocated the plugin-under-test out of the WordPress plugins tree, breaking composer autoloading of plugin classes inside the Playground runtime. Downstream, **every** data-machine PR's Test job went red with:
```
wordpress.phpunit could not run: Error: Class "DataMachine\Core\OAuth\BaseAuthProvider" not found
at /tmp/wp-codebox-inputs/0-data-machine-.../tests/Unit/Abilities/AuthAbilitiesTest.php:17
```
Timeline:
- **2026-07-02**: data-machine Test passed — 1370 tests green (built from `@main` source-install fallback).
- **2026-07-03 ~00:34 UTC**: #1698 merged to wp-codebox main.
- **2026-07-03**: identical install path, Test now crashes at collection. The **only** variable that changed was `wp-codebox@main`.
Root cause was fixed in #1699 (merged) — but the point of this issue is the **delivery mechanism that let a single upstream merge break all consumers at once**, not that specific regression.
## Why it happens
CI setup logs show the artifact-missing fallback explicitly:
```
Installing WP Codebox CLI from release artifact (.../wp-codebox-cli-linux-x64.tar.gz)...
WP Codebox release artifact not published at ...; falling back to source install
Installing WP Codebox CLI (https://github.com/Automattic/wp-codebox.git@main)...
```
The install ref defaults to `main` (`HOMEBOY_WP_CODEBOX_REF:-main` in the homeboy-extensions setup path), and consumers frequently **cannot override it** (e.g. a reusable GitHub workflow invoked via `uses:` does not inherit the caller's `env:`, and no codebox-ref input is exposed). So consumers are structurally stuck on `@main` with no pin lever.
## Requested fix (any of these breaks the "everyone tracks main" coupling)
1. **Publish versioned release artifacts** for each tagged release (e.g. `wp-codebox-cli-linux-x64.tar.gz` per `vX.Y.Z`) so the primary install path is a pinned artifact, not a source build of tip-of-main.
2. **Default the source-install fallback to the latest published release tag** rather than `main`, so a missing artifact doesn't silently escalate to bleeding-edge.
3. **Expose a first-class ref/version input** on the setup path so consumers can pin (and reusable-workflow consumers can pass it through), rather than relying on a caller `env:` that reusable workflows drop.
## Impact if unaddressed
Every future regression merged to `wp-codebox@main` will again break all downstream CI at once, with no per-consumer insulation — forcing each consumer into ad-hoc stopgaps (e.g. dropping the affected CI leg) instead of pinning to a known-good release. This is a systemic supply-chain fragility, not a one-off.
Contributor guide
No contributing guide indexed for this repository
Research direction
Trace the homeboy-extensions setup path described in the issue, including the HOMEBOY_WP_CODEBOX_REF:-main fallback and reusable GitHub workflow inputs. Check how release artifacts are selected and published, then define done as consumers using a versioned artifact or explicitly passable ref instead of silently installing @main.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions
- Domain
- ci-cd, devops, release
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100