Automattic / Automattic/wp-codebox

browser-probe: console/errors capture returns 0 on host CLI build (page console + uncaught errors not recorded)

Aperta
#632 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
TypeScript
Stelle
16
Fork
4
Merge medio
59m
PR unite (30g)
131

Descrizione

## Summary

`wordpress.browser-probe` with `capture=console,errors,screenshot` reliably produces a **screenshot**, but `console.jsonl` and `errors.jsonl` come back **empty** and `summary.json.summary` reports `consoleMessages: 0, errors: 0` regardless of what the page actually logs or throws.

## Repro

Recipe (Playground backend, WP 6.9) that:
1. seeds a published page containing a `` which both `console.log(...)`s a marker and triggers an **uncaught** `TypeError` (calling an undefined function during the page's load-time JS), then
2. runs:

```json
{
"command": "wordpress.browser-probe",
"args": [
"url=/?pagename=<seeded-page>",
"wait-for=load",
"duration=2s",
"capture=console,errors,screenshot"
]
}
```

Observed `summary.json`:

```json
{
"schema": "wp-codebox/browser-probe/v1",
"capture": ["console", "errors", "screenshot"],
"summary": { "consoleMessages": 0, "errors": 0, "screenshot": true }
}
```

`files/browser/console.jsonl` and `files/browser/errors.jsonl` are both empty even though:
- the page logs a marker repeatedly on a `setInterval` (so messages fire well within the 2s post-load window), and
- a deliberately broken variant throws an **uncaught** page error at load.

Both the success and the broken variant report identical `errors: 0` / `consoleMessages: 0`, so the probe cannot currently distinguish a clean render from one with uncaught page errors.

## Impact

Page-error / console capture is the load-bearing signal for browser smokes that assert client-side behaviour (e.g. verifying that one widget's broken callback does not abort an implicit auto-render loop for sibling widgets). With these channels inert, `browser-probe` can only prove the page loaded + screenshotted, not assert on console/errors.

## Likely area

Probably a listener-attachment ordering issue: the Playwright `page.on('console')` / `page.on('pageerror')` handlers may be attached after navigation completes, missing load-time events, and/or not being flushed into the artifact files. The screenshot path works, so the runtime + artifact plumbing is otherwise healthy.

## Environment

- Host `wp-codebox` CLI (global install), Playground backend, PHP 8.3 / WordPress 6.9.
- WP plugin side is at v0.5.0; the `wordpress.browser-actions` command (richer expect/assert steps from the v0.5.0 cookbook) is also not present in this CLI build's `wp-codebox commands` list — possibly the same packaging lag.

## Workaround in the meantime

Consumers can gate the full assertion on `summary.consoleMessages > 0` and fall back to a load+screenshot-only smoke when capture is inert, to avoid false greens.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Inizia dal punto di ingresso del comando wordpress.browser-probe e segui come gli eventi console e pageerror di Playwright vengono associati e scaricati nei file files/browser/console.jsonl e files/browser/errors.jsonl. Esegui nuovamente la recipe Playground fornita, quindi verifica che summary.json riporti il messaggio della console acquisito e l'errore non gestito, mentre l'acquisizione dello screenshot continua a funzionare.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
playwright, typescript
Ambito
testing-qa
Tipo di issue
Bug
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Tranquilla
Chiarezza
Abbastanza chiara
Idoneità per principianti
52/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.