pestphp / pestphp/pest

[Bug]: count-based --shard fallback selects zero tests on large suites (PCRE backtrack limit in --filter)

Open Beginner friendly
#1,722 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
11.7k
Forks
538
Avg merge
4d 11h
Merged PRs (30d)
8

Description

What happened?

On a large suite (~1700+ test classes), running pest --shard=1/4 without a tests/.pest/shards.json timings file makes shard 1 report No tests found and exit 1, even though hundreds of classes were assigned to it. Other shards run normally.

Root cause

In src/Plugins/Shard.php::handleArguments, when no timings file exists, tests are split into contiguous chunks:

// line 135
$testsToRun = (array_chunk($tests, max(1, (int) ceil(count($tests) / $total))))[$index - 1] ?? [];

and each chunk becomes a single |-joined regex:

// line 215
return addslashes(implode('|', $testsToRun));

Because --list-tests returns classes in discovery/alphabetical order, the first contiguous chunk is a cluster of classes sharing long common prefixes (e.g. Tests\Feature\Api\Admin\…). Joined into one unanchored alternation (~36 KB, ~436 alternatives), matching it triggers catastrophic backtracking: preg_match returns false with preg_last_error() === PREG_INTERNAL_ERROR. PHPUnit treats the errored filter as "matched nothing", so the shard runs zero tests and exits 1.

Why a committed shards.json hides it

The timings path distributes new (untimed) tests round-robin:

// line 128
$partitions[$i % $total][] = $test;

Interleaving scatters the clustered prefixes across shards, so no single filter is large/homogeneous enough to hit the limit. Any present timings file — even an empty {"timings":{}} — therefore avoids the bug; only the no-cache fallback is affected.

How to reproduce it
  1. A suite with enough classes that ceil(total/shards) exceeds ~400 contiguous prefix-clustered classes.
  2. Ensure tests/.pest/shards.json does not exist.
  3. Run pest --shard=1/4.
  4. Shard 1 → No tests found, exit 1.

Minimal confirmation of the mechanism:

$pat = '/'.addslashes(implode('|', $first436ContiguousClasses)).'/';
preg_match($pat, $someClass.'::x'); // false
echo preg_last_error();             // 1 (PREG_INTERNAL_ERROR)
// the same 436 classes interleaved → matches fine
Suggested fix

Make the no-cache fallback interleave like the new-tests path instead of using contiguous array_chunk — e.g. round-robin assignment ($i % $total). That breaks up the prefix clusters and keeps each shard's filter within PCRE limits. (Chunking/anchoring the emitted --filter, or writing it to a --filter file, would also help.)

Package Version

4.7.2 (fallback code identical since at least 4.3.2)

PHP Version

8.3

Operation System

Linux / macOS

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/Plugins/Shard.php::handleArguments and inspect the no-cache fallback around the array_chunk assignment, comparing it with the round-robin timed-tests path. Reproduce with a large suite and pest --shard=1/4 without tests/.pest/shards.json. Done means shard 1 no longer reports No tests found when classes are assigned, while the existing shard behavior remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
cli, testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.