Automattic / Automattic/wp-codebox
PHPUnit runner treats concrete helper TestCase classes as warning tests
- Lingua principale
- TypeScript
- Stelle
- 17
- Fork
- 4
- Merge medio
- 59m
- PR unite (30g)
- 131
Descrizione
## Problem
The generated plugin and core PHPUnit runners add every newly declared, non-abstract `PHPUnit\Framework\TestCase` subclass to the suite. Concrete helper/base classes that intentionally contain no runnable tests are therefore converted by PHPUnit 9 into warning tests:
```text
Warning
No tests found in class "A8C\CDN\Tests\TestCase".
```
This breaks valid suites under TeamCity's `--fail-on-warning` contract. It also diverges from PHPUnit configuration discovery, where helper classes may coexist with executable test classes in discovered files.
The affected code is in `packages/runtime-playground/src/phpunit-command-handlers.ts`, in both the plugin and WordPress core class-loading loops.
Exact WPCOM comparator evidence at `7ae9ae21ca30f7fc9724aecfb54a16f674f82a23` reproduced warnings for:
- `BkismetBaseTestCase`
- `A8C\CDN\Tests\TestCase`
- `A8C\GlotPressPlugins\Deliverables\DeliverablesTestCase`
- `GlotPressPluginsTestCase`
- `A8C\Test\Integration\WPCOM\Legacy\Upgrades\BaseAssignPayPalPaymentMethod`
WPCOM's hierarchy adds a second edge case: base classes expose an inherited static `suite()` factory. Checking only `hasMethod('suite')` still admits every concrete helper. A suite factory should qualify the candidate only when the candidate class declares it; normal inherited executable test methods must still qualify.
## Reproduction
Place a concrete helper beside a real test:
```php
class ConcreteTestHelper extends WP_UnitTestCase {
public function fixture(): string {
return 'fixture';
}
}
class ExampleTest extends WP_UnitTestCase {
public function test_example(): void {
$this->assertTrue(true);
}
}
```
Run the generated `wordpress.phpunit` recipe with warning failures enabled. The real test passes, but the helper creates `No tests found in class "ConcreteTestHelper"` and fails the suite.
## Expected
The generated runner should schedule only classes that PHPUnit recognizes as containing executable tests, plus candidate-owned static suite factories. Concrete helper classes should remain loadable without becoming warning tests.
## Acceptance criteria
- Concrete `TestCase` subclasses with no runnable tests are not added as suites.
- Abstract test bases remain skipped.
- `test*` methods, `@test` methods, and inherited runnable test methods remain discoverable.
- A static `suite()` declared by the candidate class remains supported.
- An inherited generic `suite()` does not qualify an otherwise empty helper class.
- Plugin and WordPress core generated runners share the same behavior.
- Unit coverage exercises both public and private PHPUnit `TestSuite` constructors supported by the runner.
- A real PHPUnit 9.6 Playground integration proves a helper and executable class can coexist without warnings and without mutating readonly source.
## Evidence
A local candidate on PR #1938's branch produced these exact-SHA results:
- `isolated:cdn`: failed before, passed after.
- `isolated:glotpress-plugins`: failed before, passed after.
- Nine-suite warning cluster: improved from 3/9 to 5/9, with all `No tests found in class` diagnostics removed and zero timeouts.
## Related
- #1937 owns clean-process bounded PHPUnit execution.
- PR #1938 implements that path and is where this defect surfaced.
## AI assistance
- **Model:** OpenAI GPT-5.6 Sol
- **Tool:** OpenCode
- **Used for:** Root-cause analysis, implementation and regression coverage, real PHPUnit integration verification, exact WPCOM comparator runs, and issue drafting under Chris Huber's direction.
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Direzione di ricerca
Inizia da packages/runtime-playground/src/phpunit-command-handlers.ts e confronta i cicli di caricamento delle classi del plugin e del core di WordPress. Esegui la recipe wordpress.phpunit generata con l’abilitazione degli errori per gli avvisi, quindi aggiungi la copertura dei test unitari per i costruttori pubblici e privati di PHPUnit TestSuite e un’integrazione Playground con PHPUnit 9.6. Il lavoro è completo quando le classi helper coesistono con test eseguibili senza avvisi e continuano a essere supportati sia i metodi eseguibili ereditati sia le factory di suite dichiarate dal candidato.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- php, typescript
- Ambito
- testing-qa, tooling
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Tranquilla
- Chiarezza
- Specificata chiaramente
- Idoneità per principianti
- 62/100