Automattic / Automattic/wp-codebox
browser-probe: console/errors capture returns 0 on host CLI build (page console + uncaught errors not recorded)
- Dominant language
- TypeScript
- Stars
- 16
- Forks
- 4
- Avg merge
- 59m
- Merged PRs (30d)
- 131
Description
## Summary
`wordpress.browser-probe` with `capture=console,errors,screenshot` reliably produces a **screenshot**, but `console.jsonl` and `errors.jsonl` come back **empty** and `summary.json.summary` reports `consoleMessages: 0, errors: 0` regardless of what the page actually logs or throws.
## Repro
Recipe (Playground backend, WP 6.9) that:
1. seeds a published page containing a `` which both `console.log(...)`s a marker and triggers an **uncaught** `TypeError` (calling an undefined function during the page's load-time JS), then
2. runs:
```json
{
"command": "wordpress.browser-probe",
"args": [
"url=/?pagename=<seeded-page>",
"wait-for=load",
"duration=2s",
"capture=console,errors,screenshot"
]
}
```
Observed `summary.json`:
```json
{
"schema": "wp-codebox/browser-probe/v1",
"capture": ["console", "errors", "screenshot"],
"summary": { "consoleMessages": 0, "errors": 0, "screenshot": true }
}
```
`files/browser/console.jsonl` and `files/browser/errors.jsonl` are both empty even though:
- the page logs a marker repeatedly on a `setInterval` (so messages fire well within the 2s post-load window), and
- a deliberately broken variant throws an **uncaught** page error at load.
Both the success and the broken variant report identical `errors: 0` / `consoleMessages: 0`, so the probe cannot currently distinguish a clean render from one with uncaught page errors.
## Impact
Page-error / console capture is the load-bearing signal for browser smokes that assert client-side behaviour (e.g. verifying that one widget's broken callback does not abort an implicit auto-render loop for sibling widgets). With these channels inert, `browser-probe` can only prove the page loaded + screenshotted, not assert on console/errors.
## Likely area
Probably a listener-attachment ordering issue: the Playwright `page.on('console')` / `page.on('pageerror')` handlers may be attached after navigation completes, missing load-time events, and/or not being flushed into the artifact files. The screenshot path works, so the runtime + artifact plumbing is otherwise healthy.
## Environment
- Host `wp-codebox` CLI (global install), Playground backend, PHP 8.3 / WordPress 6.9.
- WP plugin side is at v0.5.0; the `wordpress.browser-actions` command (richer expect/assert steps from the v0.5.0 cookbook) is also not present in this CLI build's `wp-codebox commands` list — possibly the same packaging lag.
## Workaround in the meantime
Consumers can gate the full assertion on `summary.consoleMessages > 0` and fall back to a load+screenshot-only smoke when capture is inert, to avoid false greens.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.