[Bug]: [mutate] Windows: quoted --filter breaks cmd when a line is covered by several tests, so every mutant counts as killed
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 11.7k
- Forks
- 538
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 8
Description
What Happened
On Windows, pest --mutate reports every mutant as killed (score 100%) whenever the mutated line is covered by more than one test. This happens even when the per-mutant child process launches correctly, so it's separate from the forward-slash argv[0] problem in #1916.
MutationTest::start() wraps the filter in literal quotes:
'--filter="'.implode('|', $filters).'"',
The command is an array, so Symfony Process already escapes each argument. On Windows it doubles the embedded ", which ends the quoted section for cmd.exe, so the | between two filters is parsed as a pipe:
'TtyTest::' is not recognized as an internal or external command,
operable program or batch file.
The child exits non-zero, hasFinished() maps that to Tested, and the mutant counts as killed.
Expected: the child runs the covering tests, and a mutant that no test kills is reported as untested.
How to Reproduce
Isolated with the child command built the way the plugin builds it. argv[0] is the absolute path Composer's .bat proxy passes, which launches fine:
use Symfony\Component\Process\Process;
$filter = 'TtyTest::(.*)it.runs.a.binary.from.bin.dir|TtyTest::(.*)it.passes.the.exit.code.through';
$process = new Process([getcwd().'\vendor\bin\/pest', '--bail', '--filter="'.$filter.'"']);
$process->run();
// exit 255: 'TtyTest::' is not recognized as an internal or external command
$process = new Process([getcwd().'\vendor\bin\/pest', '--bail', '--filter='.$filter]);
$process->run();
// exit 0: the 4 matching tests run and pass
End to end, with the sample repository on Windows (PHP 8.5 with Xdebug):
composer install- In
src/Tty.php, delete the two// @pest-mutate-ignore: RemoveArrayItemcomments. That leaves 2 mutants no test can kill. composer test:mutatereports 100%. On Linux, the same run reports the 2 survivors (94.87%).
Suggested fix: drop the literal quotes, '--filter='.implode('|', $filters),. The process escapes the argument itself.
Sample Repository
https://github.com/Shoxcie/composer-tty
Pest Version
5.2.1 (pestphp/pest-plugin-mutate v5.0.2, symfony/process v8.1.7)
PHP Version
8.5.10
Operation System
Windows
Notes
Filed here because issues are disabled on pestphp/pest-plugin-mutate.
Related: #1916 (forward-slash argv[0]) and #1313 (Windows absolute paths in <source>). This bug remains after working around both. The sample repository's test:mutate runs through Composer, so argv[0] launches, and it passes --path=src, so the files are found. Yet every mutant still counts as killed. As #1916 points out, the deeper problem is that any failed child run counts as a kill, so a broken command silently produces a perfect score.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at MutationTest::start(), where the mutation command builds its --filter argument, and compare the quoted and unquoted Windows command forms shown in the issue. Run the provided composer-tty reproduction on Windows; done means covering tests execute successfully and mutants that no test kills are reported as survivors rather than all being counted as killed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php, symfony
- Domain
- cli, testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 80/100