Automattic / Automattic/wp-codebox
recipe-run exits without a parseable summary on RuntimeServiceProvisionError; consumers report zero tests instead of the cause
- Dominant language
- TypeScript
- Stars
- 16
- Forks
- 4
- Avg merge
- 59m
- Merged PRs (30d)
- 131
Description
## Summary
When a managed runtime service fails to provision (Docker `ENOENT`, or `external` provider with unresolved `secret_env`), `recipe-run` exits without flushing its JSON summary. Every downstream consumer (Homeboy's `wp-codebox-phpunit-adapter.mjs`) then classifies the run as `recipe_run_payload_unparseable` and reports zero executed tests, hiding the real cause. Homeboy has six open issues that are all this symptom (Extra-Chill/homeboy #12683 #12776 #12784 #12795 #13132 #13136, now consolidated under #14449) and this repo has #2327 ("intermittently returns unparseable payload").
## Evidence
From Extra-Chill/data-machine-events#795 (Docker-free host, before the provider switch):
```
RuntimeServiceProvisionError: Managed runtime service failed: wordpress-database
provider: 'docker', diagnostic: { code: 'provider-unavailable', command: 'docker', cause: { code: 'ENOENT' } }
```
Consumer side (Homeboy adapter, `wordpress/scripts/test/wp-codebox-phpunit-adapter.mjs:1473`):
```
PHPUNIT_ZERO_TESTS cause=recipe_run_payload_unparseable
The recipe-run JSON payload could not be parsed, so no execution step ledger is available.
Read logs/recipe-run.stdout.log and logs/recipe-run.stderr.log for the raw recipe-run output; a crashed or truncated recipe-run may not have flushed its JSON summary.
```
Same result today on the same host with `provider: external` and `WP_CODEBOX_DB_*` unset.
## Expected
`recipe-run` always emits a parseable summary, including on early failure. On `RuntimeServiceProvisionError` (and any pre-execution fatal), the payload should carry `parse_status`-compatible structure with an empty `executions` ledger plus a structured `failure` block: `{ phase: "runtime_services", service: "wordpress-database", provider, diagnostic: { code, command?, cause? } }`. Exit code non-zero as today.
Where: `packages/cli/src/runtime-services.ts` raises `RuntimeServiceProvisionError`; the recipe-run summary writer needs a `try/finally` (or top-level catch) that serializes the error into the summary before exit. Add a test alongside `tests/runtime-services.test.ts` asserting the summary is written and parseable when provisioning throws.
Refs #2327.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with packages/cli/src/runtime-services.ts and the recipe-run summary writer, then read tests/runtime-services.test.ts. Reproduce a RuntimeServiceProvisionError during provisioning and verify that recipe-run still writes a parseable summary with an empty executions ledger, structured runtime-services failure details, and a non-zero exit code.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, typescript
- Domain
- backend, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100