Automattic / Automattic/wp-codebox

Recipe run intermittently returns unparseable payload under PHPUnit discovery

Open
#2,327 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
16
Forks
4
Avg merge
59m
Merged PRs (30d)
131

Description

## Problem

The supported Homeboy WordPress PHPUnit adapter repeatedly receives an unparseable `wp-codebox recipe-run` payload during canonical discovery, yielding zero tests before PHPUnit starts.

Latest production release preflight reproduction on 2026-08-23:

```text
Running PHPUnit tests via WP Codebox...
Plugin: data-machine-events
Backend: wp-codebox
PHPUNIT_ZERO_TESTS cause=recipe_run_payload_unparseable
The recipe-run JSON payload could not be parsed, so no execution step ledger is available.
Read logs/recipe-run.stdout.log and logs/recipe-run.stderr.log for the raw recipe-run output; a crashed or truncated recipe-run may not have flushed its JSON summary.
```

This blocked `homeboy release data-machine-events` at `preflight.test`. The same consumer suite executes in GitHub CI (808 tests), so the failure occurs before consumer PHPUnit execution. Homeboy Extensions #2650 improved evidence for this failure class, and #2668 fixed a separate bounded-text reader crash; neither fixes the producer payload.

The host was under elevated load during the latest reproduction, which may be relevant if recipe-run output is truncated or a subprocess exits before flushing the final JSON envelope.

## Expected behavior

`wp-codebox recipe-run` should always emit one complete, parseable terminal JSON envelope, including structured failure details when setup or a child process fails. Diagnostic text must not contaminate stdout.

## Investigation

- Audit every recipe-run exit path for partial/unflushed JSON and stdout contamination.
- Reproduce under concurrent/high-load PHPUnit discovery.
- Ensure child crashes, timeouts, and process termination still produce a terminal structured envelope.
- Add a stress/regression test that repeatedly parses recipe-run stdout while setup/test children fail at different phases.

Related evidence:
- https://github.com/Extra-Chill/homeboy-extensions/issues/2650
- https://github.com/Extra-Chill/homeboy-extensions/issues/2667

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the wp-codebox recipe-run entry point and inspect logs/recipe-run.stdout.log and logs/recipe-run.stderr.log from the reported failure. Trace each exit path, then reproduce under concurrent or high-load PHPUnit discovery. Done means stdout always contains one complete parseable terminal JSON envelope, with diagnostics kept out of stdout, and the repeated stress/regression test passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
php, typescript
Domain
cli, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.