[Bug]: Using `--parallel` causes overall process to fail with no reporting as to why even though all tests pass
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 11.7k
- Forks
- 538
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 8
Description
What Happened
Our team noticed after upgrading to Pest 4 that our tests running in parallel are showing all tests as passing, but having the overall command fail. However, if we run the exact same test suite sequentially, we get the same number of passing tests and the overall command passes.
Sequential
./vendor/bin/pest --exclude-group=fails-in-parallel --testsuite=Feature --compact
................................................................................................................................................................
............................................................................
Tests: 236 passed (364 assertions)
Duration: 139.51s
echo $?
0
Parallel
./vendor/bin/pest --exclude-group=fails-in-parallel --testsuite=Feature --parallel
................................................................................................................................................................
............................................................................
Tests: 236 passed (364 assertions)
Duration: 30.20s
Parallel: 12 processes
echo $?
2
How to Reproduce
- Run test suite without
--parallelflag and see it passes with the same number of tests and assertions - Run test suite with
--parallelflag and see it returns an error code of 2, even though it shows the same number of tests and assertions passing successfully
Sample Repository
No response
Pest Version
4.0.3
PHP Version
8.4.11
Operation System
macOS
Notes
I'm not certain if it's related, but I've also noticed that when we run tests in parallel now and there is a test failure, it is swallowing the error and immediately killing the tests without reporting anything about what caused the failure. There was a previous issue #1451 that hinted at their problem could be related to the master process not properly handling the exit codes of the worker threads, which feels like it could be a reason for both of these issues.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with Pest's --parallel execution path and the master process's handling of worker exit codes; issue #1451 is the related report mentioned in the issue. Reproduce with the sequential and parallel commands against Pest 4.0.3, then verify that parallel failures and nonzero exits report their cause instead of silently terminating.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- cli, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100