Automattic / Automattic/wp-codebox

fix: honor project bootstrap before plugin test discovery

Open
#2,246 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
16
Forks
4
Avg merge
59m
Merged PRs (30d)
131

Description

## Problem

`wordpress.phpunit` in project mode discovers plugin test files before executing the bootstrap declared by the mounted project's `phpunit.xml`. A repository whose tests depend on bootstrap-owned fixtures crashes before any selected tests execute.

This regresses the project-bootstrap contract completed in #644 and the harness-before-discovery fix completed in #1771.

## Reproduction

From `Extra-Chill/extrachill-artist-platform` PR #171 at commit `ac7ea32`:

```bash
homeboy review test extrachill-artist-platform \
--path /var/lib/datamachine/workspace/extrachill-artist-platform@refactor-170-owner-neutral-operations \
--changed-since origin/main \
--placement local \
-- --filter LinkPageOperationsTest
```

Homeboy correctly scopes the request to:

- `tests/AbilityAuthorizationTest.php`
- `tests/LinkPageOperationsTest.php`
- `tests/LinkPageOwnerReferenceTest.php`

The Codebox runtime still discovers `tests/ArtistDirectSubscriberTest.php` and crashes before running any tests:

```text
wordpress.phpunit crashed before producing a structured response
Error: Class \"EcTestWpdb\" not found at
/wordpress/wp-content/plugins/extrachill-artist-platform/tests/ArtistDirectSubscriberTest.php:5
[test-results] Total: 0, Passed: 0, Failed: 0, Skipped: 0
```

The class exists in `tests/bootstrap.php`, and the repository's local PHPUnit command honors that bootstrap: 246 tests / 1,330 assertions pass. Earlier aggregate runs also reported missing `AgentsAPI\\AI\\WP_Agent_Execution_Principal`, `get_site()`, and `switch_to_blog()`, all defined by the same project bootstrap.

Persisted Homeboy runs:

- aggregate changed-file run: `defad187-aa51-498a-95fe-bc0387df14cd`
- explicit filtered run: `3694ac54-17ab-4e3b-b4c4-fb71f19aa072`

## Expected

1. Read the mounted project's PHPUnit configuration before test discovery.
2. Execute its declared bootstrap before loading any test class.
3. Restrict discovery to the selected files/filter when a focused workload is requested.
4. Return structured zero/failure evidence only after the project bootstrap and selected test workload actually run.

## Acceptance criteria

- A fixture class defined only in `tests/bootstrap.php` is available to every discovered test.
- `--filter LinkPageOperationsTest` does not discover unrelated plugin test classes first.
- The Extra Chill Artist Platform reproduction executes its selected tests instead of reporting zero tests.

## AI assistance

- **AI assistance:** Yes
- **Tool:** OpenCode
- **Used for:** Reproducing the failure, inspecting persisted Codebox artifacts, and drafting this report.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the wordpress.phpunit entry point and the mounted project's phpunit.xml, then trace when tests/bootstrap.php is loaded relative to test discovery. Run the provided Homeboy review command with --filter LinkPageOperationsTest and verify that bootstrap fixtures are available, unrelated plugin tests are not discovered first, and selected tests produce structured results.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.