[Bug]: count-based --shard fallback selects zero tests on large suites (PCRE backtrack limit in --filter)
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 11.7k
- Forks
- 538
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 8
Description
What happened?
On a large suite (~1700+ test classes), running pest --shard=1/4 without a tests/.pest/shards.json timings file makes shard 1 report No tests found and exit 1, even though hundreds of classes were assigned to it. Other shards run normally.
Root cause
In src/Plugins/Shard.php::handleArguments, when no timings file exists, tests are split into contiguous chunks:
// line 135
$testsToRun = (array_chunk($tests, max(1, (int) ceil(count($tests) / $total))))[$index - 1] ?? [];
and each chunk becomes a single |-joined regex:
// line 215
return addslashes(implode('|', $testsToRun));
Because --list-tests returns classes in discovery/alphabetical order, the first contiguous chunk is a cluster of classes sharing long common prefixes (e.g. Tests\Feature\Api\Admin\…). Joined into one unanchored alternation (~36 KB, ~436 alternatives), matching it triggers catastrophic backtracking: preg_match returns false with preg_last_error() === PREG_INTERNAL_ERROR. PHPUnit treats the errored filter as "matched nothing", so the shard runs zero tests and exits 1.
Why a committed shards.json hides it
The timings path distributes new (untimed) tests round-robin:
// line 128
$partitions[$i % $total][] = $test;
Interleaving scatters the clustered prefixes across shards, so no single filter is large/homogeneous enough to hit the limit. Any present timings file — even an empty {"timings":{}} — therefore avoids the bug; only the no-cache fallback is affected.
How to reproduce it
- A suite with enough classes that
ceil(total/shards)exceeds ~400 contiguous prefix-clustered classes. - Ensure
tests/.pest/shards.jsondoes not exist. - Run
pest --shard=1/4. - Shard 1 →
No tests found, exit 1.
Minimal confirmation of the mechanism:
$pat = '/'.addslashes(implode('|', $first436ContiguousClasses)).'/';
preg_match($pat, $someClass.'::x'); // false
echo preg_last_error(); // 1 (PREG_INTERNAL_ERROR)
// the same 436 classes interleaved → matches fine
Suggested fix
Make the no-cache fallback interleave like the new-tests path instead of using contiguous array_chunk — e.g. round-robin assignment ($i % $total). That breaks up the prefix clusters and keeps each shard's filter within PCRE limits. (Chunking/anchoring the emitted --filter, or writing it to a --filter file, would also help.)
Package Version
4.7.2 (fallback code identical since at least 4.3.2)
PHP Version
8.3
Operation System
Linux / macOS
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/Plugins/Shard.php::handleArguments and inspect the no-cache fallback around the array_chunk assignment, comparing it with the round-robin timed-tests path. Reproduce with a large suite and pest --shard=1/4 without tests/.pest/shards.json. Done means shard 1 no longer reports No tests found when classes are assigned, while the existing shard behavior remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- cli, testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100