Automattic / Automattic/wp-codebox

Benchmark aggregation reports skipped workloads as successful timing samples

Aperta
#2,495 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
TypeScript
Stelle
17
Fork
4
Merge medio
59m
PR unite (30g)
131

Descrizione

## Observed failure
The MDI decision matrix completed with PASS for both backends, including a boot-timing workload that returned `skipped: missing_bench_boot_phase`. Aggregation reported approximately 0.004 ms as its duration, presenting the skip path as a benchmark result rather than a skipped scenario.

Reproduction workload: Automattic/markdown-database-integration, `tests/bench/boot-timing.php`, matrix source `21267e18d41c88396e4883e227422158ba5c16b2`. Invoke the decision benchmark profile without BENCH_BOOT_PHASE. Operator evidence: Homeboy job `7018d7ce-fdf0-442d-a95c-ad96eedac2c5`, SQLite run `b28d11dd-e33d-40cf-8944-683aacb5fbc3`, native run `2cc3b041-9291-4e8b-823f-e05b92376607`. The source and normal workload invocation are the portable reproduction; IDs require operator access.

## Expected contract
The workload result boundary preserves typed skip status and reason. Skipped workloads contribute no performance sample and are explicitly represented in matrix completeness. A required skipped scenario leaves acceptance incomplete, even when other executed scenarios pass. Test aggregation with executed, skipped, and failed callables through the real dispatcher.

Inspect owning layers before implementation: if WP Codebox already preserves skip status and the loss occurs downstream, route the repair to that aggregator rather than duplicating logic.

## AI Assistance
GPT-6 Astra through OpenCode inspected the benchmark output and workload source, searched existing trackers, and drafted this report under Chris Huber’s direction. Direct OpenCode repair runs use openai/gpt-5.6-terra.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Riproduci il problema con tests/bench/boot-timing.php tramite il profilo di benchmark delle decisioni senza BENCH_BOOT_PHASE, quindi segui il risultato attraverso il dispatcher reale e il livello di aggregazione responsabile. Esegui test su Callable eseguiti, saltati e falliti e verifica che i workload saltati non producano alcun campione di timing, conservino il motivo e lascino incompleta la completezza richiesta della matrice.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
php, typescript, wordpress
Ambito
backend, performance, testing
Tipo di issue
Bug
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Attiva
Chiarezza
Abbastanza chiara
Idoneità per principianti
68/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.