Automattic / Automattic/wp-codebox

recipe-run exits without a parseable summary on RuntimeServiceProvisionError; consumers report zero tests instead of the cause

Open
#2,486 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
16
Forks
4
Avg merge
59m
Merged PRs (30d)
131

Description

## Summary

When a managed runtime service fails to provision (Docker `ENOENT`, or `external` provider with unresolved `secret_env`), `recipe-run` exits without flushing its JSON summary. Every downstream consumer (Homeboy's `wp-codebox-phpunit-adapter.mjs`) then classifies the run as `recipe_run_payload_unparseable` and reports zero executed tests, hiding the real cause. Homeboy has six open issues that are all this symptom (Extra-Chill/homeboy #12683 #12776 #12784 #12795 #13132 #13136, now consolidated under #14449) and this repo has #2327 ("intermittently returns unparseable payload").

## Evidence

From Extra-Chill/data-machine-events#795 (Docker-free host, before the provider switch):
```
RuntimeServiceProvisionError: Managed runtime service failed: wordpress-database
provider: 'docker', diagnostic: { code: 'provider-unavailable', command: 'docker', cause: { code: 'ENOENT' } }
```
Consumer side (Homeboy adapter, `wordpress/scripts/test/wp-codebox-phpunit-adapter.mjs:1473`):
```
PHPUNIT_ZERO_TESTS cause=recipe_run_payload_unparseable
The recipe-run JSON payload could not be parsed, so no execution step ledger is available.
Read logs/recipe-run.stdout.log and logs/recipe-run.stderr.log for the raw recipe-run output; a crashed or truncated recipe-run may not have flushed its JSON summary.
```
Same result today on the same host with `provider: external` and `WP_CODEBOX_DB_*` unset.

## Expected

`recipe-run` always emits a parseable summary, including on early failure. On `RuntimeServiceProvisionError` (and any pre-execution fatal), the payload should carry `parse_status`-compatible structure with an empty `executions` ledger plus a structured `failure` block: `{ phase: "runtime_services", service: "wordpress-database", provider, diagnostic: { code, command?, cause? } }`. Exit code non-zero as today.

Where: `packages/cli/src/runtime-services.ts` raises `RuntimeServiceProvisionError`; the recipe-run summary writer needs a `try/finally` (or top-level catch) that serializes the error into the summary before exit. Add a test alongside `tests/runtime-services.test.ts` asserting the summary is written and parseable when provisioning throws.

Refs #2327.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with packages/cli/src/runtime-services.ts and the recipe-run summary writer, then read tests/runtime-services.test.ts. Reproduce a RuntimeServiceProvisionError during provisioning and verify that recipe-run still writes a parseable summary with an empty executions ledger, structured runtime-services failure details, and a non-zero exit code.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, typescript
Domain
backend, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.