Automattic / Automattic/wp-codebox

wordpress.run-php should return structured assertion failures instead of crash diagnostics for exit 1

Open
#1,014 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
16
Forks
4
Avg merge
59m
Merged PRs (30d)
131

Description

## Problem
When a PHP smoke exits non-zero because an assertion failed, WP Codebox reports:

`wordpress.run-php crashed before producing a structured response`

and wraps the failure as a Playground crash. In Data Machine PR CI this happened for normal smoke failures such as:

- `tests/agent-bundle-artifact-store-smoke.php` exiting 1 after `FAIL: secret value is not stored`
- `tests/agent-bundle-artifact-rebase-smoke.php` exiting 1 after a missing class

The stderr/stdout is present but buried inside crash diagnostics instead of a structured command result.

## Expected
`wordpress.run-php` should distinguish:

- PHP fatal/engine crash
- script assertion failure / exit 1
- structured response protocol failure

For normal non-zero script exit, return a structured result with exit code, stdout/stderr, and a concise summary so upstream runners can aggregate/report failures without treating them as sandbox crashes.

## Evidence
Data Machine PR: https://github.com/Extra-Chill/data-machine/pull/2638
Example run: https://github.com/Extra-Chill/data-machine/actions/runs/27582843874

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.