Automattic / Automattic/wp-codebox

browser-probe: console/errors capture returns 0 on host CLI build (page console + uncaught errors not recorded)

Open
#632 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
16
Forks
4
Avg merge
59m
Merged PRs (30d)
131

Description

## Summary

`wordpress.browser-probe` with `capture=console,errors,screenshot` reliably produces a **screenshot**, but `console.jsonl` and `errors.jsonl` come back **empty** and `summary.json.summary` reports `consoleMessages: 0, errors: 0` regardless of what the page actually logs or throws.

## Repro

Recipe (Playground backend, WP 6.9) that:
1. seeds a published page containing a `` which both `console.log(...)`s a marker and triggers an **uncaught** `TypeError` (calling an undefined function during the page's load-time JS), then
2. runs:

```json
{
"command": "wordpress.browser-probe",
"args": [
"url=/?pagename=<seeded-page>",
"wait-for=load",
"duration=2s",
"capture=console,errors,screenshot"
]
}
```

Observed `summary.json`:

```json
{
"schema": "wp-codebox/browser-probe/v1",
"capture": ["console", "errors", "screenshot"],
"summary": { "consoleMessages": 0, "errors": 0, "screenshot": true }
}
```

`files/browser/console.jsonl` and `files/browser/errors.jsonl` are both empty even though:
- the page logs a marker repeatedly on a `setInterval` (so messages fire well within the 2s post-load window), and
- a deliberately broken variant throws an **uncaught** page error at load.

Both the success and the broken variant report identical `errors: 0` / `consoleMessages: 0`, so the probe cannot currently distinguish a clean render from one with uncaught page errors.

## Impact

Page-error / console capture is the load-bearing signal for browser smokes that assert client-side behaviour (e.g. verifying that one widget's broken callback does not abort an implicit auto-render loop for sibling widgets). With these channels inert, `browser-probe` can only prove the page loaded + screenshotted, not assert on console/errors.

## Likely area

Probably a listener-attachment ordering issue: the Playwright `page.on('console')` / `page.on('pageerror')` handlers may be attached after navigation completes, missing load-time events, and/or not being flushed into the artifact files. The screenshot path works, so the runtime + artifact plumbing is otherwise healthy.

## Environment

- Host `wp-codebox` CLI (global install), Playground backend, PHP 8.3 / WordPress 6.9.
- WP plugin side is at v0.5.0; the `wordpress.browser-actions` command (richer expect/assert steps from the v0.5.0 cookbook) is also not present in this CLI build's `wp-codebox commands` list — possibly the same packaging lag.

## Workaround in the meantime

Consumers can gate the full assertion on `summary.consoleMessages > 0` and fall back to a load+screenshot-only smoke when capture is inert, to avoid false greens.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.