Automattic / Automattic/wp-codebox
PHPUnit runner treats concrete helper TestCase classes as warning tests
- Vorherrschende Sprache
- TypeScript
- Sterne
- 17
- Forks
- 4
- Ø Merge
- 59 Min.
- Gemergte PRs (30 T.)
- 131
Beschreibung
## Problem
The generated plugin and core PHPUnit runners add every newly declared, non-abstract `PHPUnit\Framework\TestCase` subclass to the suite. Concrete helper/base classes that intentionally contain no runnable tests are therefore converted by PHPUnit 9 into warning tests:
```text
Warning
No tests found in class "A8C\CDN\Tests\TestCase".
```
This breaks valid suites under TeamCity's `--fail-on-warning` contract. It also diverges from PHPUnit configuration discovery, where helper classes may coexist with executable test classes in discovered files.
The affected code is in `packages/runtime-playground/src/phpunit-command-handlers.ts`, in both the plugin and WordPress core class-loading loops.
Exact WPCOM comparator evidence at `7ae9ae21ca30f7fc9724aecfb54a16f674f82a23` reproduced warnings for:
- `BkismetBaseTestCase`
- `A8C\CDN\Tests\TestCase`
- `A8C\GlotPressPlugins\Deliverables\DeliverablesTestCase`
- `GlotPressPluginsTestCase`
- `A8C\Test\Integration\WPCOM\Legacy\Upgrades\BaseAssignPayPalPaymentMethod`
WPCOM's hierarchy adds a second edge case: base classes expose an inherited static `suite()` factory. Checking only `hasMethod('suite')` still admits every concrete helper. A suite factory should qualify the candidate only when the candidate class declares it; normal inherited executable test methods must still qualify.
## Reproduction
Place a concrete helper beside a real test:
```php
class ConcreteTestHelper extends WP_UnitTestCase {
public function fixture(): string {
return 'fixture';
}
}
class ExampleTest extends WP_UnitTestCase {
public function test_example(): void {
$this->assertTrue(true);
}
}
```
Run the generated `wordpress.phpunit` recipe with warning failures enabled. The real test passes, but the helper creates `No tests found in class "ConcreteTestHelper"` and fails the suite.
## Expected
The generated runner should schedule only classes that PHPUnit recognizes as containing executable tests, plus candidate-owned static suite factories. Concrete helper classes should remain loadable without becoming warning tests.
## Acceptance criteria
- Concrete `TestCase` subclasses with no runnable tests are not added as suites.
- Abstract test bases remain skipped.
- `test*` methods, `@test` methods, and inherited runnable test methods remain discoverable.
- A static `suite()` declared by the candidate class remains supported.
- An inherited generic `suite()` does not qualify an otherwise empty helper class.
- Plugin and WordPress core generated runners share the same behavior.
- Unit coverage exercises both public and private PHPUnit `TestSuite` constructors supported by the runner.
- A real PHPUnit 9.6 Playground integration proves a helper and executable class can coexist without warnings and without mutating readonly source.
## Evidence
A local candidate on PR #1938's branch produced these exact-SHA results:
- `isolated:cdn`: failed before, passed after.
- `isolated:glotpress-plugins`: failed before, passed after.
- Nine-suite warning cluster: improved from 3/9 to 5/9, with all `No tests found in class` diagnostics removed and zero timeouts.
## Related
- #1937 owns clean-process bounded PHPUnit execution.
- PR #1938 implements that path and is where this defect surfaced.
## AI assistance
- **Model:** OpenAI GPT-5.6 Sol
- **Tool:** OpenCode
- **Used for:** Root-cause analysis, implementation and regression coverage, real PHPUnit integration verification, exact WPCOM comparator runs, and issue drafting under Chris Huber's direction.
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Rechercherichtung
Beginne in packages/runtime-playground/src/phpunit-command-handlers.ts und vergleiche die Schleifen zum Laden von Klassen im Plugin und im WordPress-Kern. Führe das generierte wordpress.phpunit-Rezept mit aktivierter Behandlung von Warnungen als Fehler aus und füge Unit-Tests sowohl für öffentliche als auch private PHPUnit TestSuite-Konstruktoren sowie eine Playground-Integration für PHPUnit 9.6 hinzu. Erledigt ist die Aufgabe, wenn Hilfsklassen neben ausführbaren Tests ohne Warnungen koexistieren und weiterhin geerbte ausführbare Methoden sowie vom Kandidaten deklarierte Suite-Factories unterstützt werden.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- php, typescript
- Bereich
- testing-qa, tooling
- Issue-Typ
- Bug
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Aktivitätsstatus
- Ruhig
- Klarheit
- Klar beschrieben
- Anfängerfreundlichkeit
- 62/100