Automattic / Automattic/wp-codebox

bug: runtime-backed run-fuzz-suite advertises checkpoint reset without handlers

Open
#2,321 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
16
Forks
4
Avg merge
59m
Merged PRs (30d)
131

Description

## Problem

The standalone `run-fuzz-suite --runner-mode=runtime-backed` surface advertises `wp-codebox.checkpoint-create` and `wp-codebox.checkpoint-restore` in runner capabilities, but a suite using `resetPolicy.mode: checkpoint-per-case` fails every case with:

```
No Playground command handler is registered for: wp-codebox.checkpoint-create
```

## Expected

The runtime-backed runner should either route reset operations through the runtime checkpoint primitives, as the recipe-backed `wp-codebox/run-fuzz-suite` path does, or stop advertising checkpoint reset support.

## Reproduction

Run a `wp-codebox/fuzz-suite/v1` suite with `resetPolicy: { "mode": "checkpoint-per-case", "checkpointName": "baseline" }` through:

```
wp-codebox run-fuzz-suite --input-file suite.json --format=json --runner-mode=runtime-backed
```

The result is `status: error`; each reset contains `fuzz_suite_reset_execution_error` with the missing-handler message above.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the standalone `run-fuzz-suite` runtime-backed runner entry point and trace how it advertises capabilities and executes `checkpoint-per-case` resets. Compare this with the recipe-backed `wp-codebox/run-fuzz-suite` path and its runtime checkpoint primitives. Done means the reproduction no longer reports a missing handler, or checkpoint reset support is no longer advertised.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
cli, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
56/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.