[Bug]: TIA records a `match` dispatch line as executable-with-zero-hits, so a replayed `--coverage` run is one line short
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 11.7k
- Forks
- 538
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 8
Description
What happened
On a replayed TIA run, the coverage report shows one line as uncovered that is genuinely covered, which fails a strict threshold on a suite that is actually at 100%.
$ pest --coverage --exactly=100.0
Tests: 1093 passed (3062 assertions, 1093 replayed)
Services/WhatsApp/WhatsAppInboundService ............ 134 / 98.8%
Total: 99.9 %
FAIL Code coverage not exactly 100.0 %, currently 99.9 %.
Line 134 is the dispatch line of a match expression whose every arm is covered:
private function body(WhatsAppMessageType $type, array $message): ?string
{
return match ($type) { // 134 — reported uncovered
WhatsAppMessageType::Text => WhatsAppValue::nullableString($message, 'text.body'), // 135
WhatsAppMessageType::Image => WhatsAppValue::nullableString($message, 'image.caption'), // 136
WhatsAppMessageType::Video => WhatsAppValue::nullableString($message, 'video.caption'), // 137
WhatsAppMessageType::Document => WhatsAppValue::nullableString($message, 'document.caption'), // 138
WhatsAppMessageType::Reaction => WhatsAppValue::nullableString($message, 'reaction.emoji'), // 139
WhatsAppMessageType::Interactive => WhatsAppValue::nullableString($message, 'interactive.button_reply.title') // 140
?? WhatsAppValue::nullableString($message, 'interactive.list_reply.title'), // 141
default => null, // 142
};
}
Reproduction
Two commands, same tree, no edits in between:
$ pest --tia --fresh --coverage # records the graph, runs all 1093 for real
Total: 100.0 %
$ pest --coverage # replays that same freshly-recorded graph
Total: 99.9 % # 1093 replayed
pest --no-tia --coverage also reports 100.0%. So only the replay is short, and it is short against a graph recorded moments earlier — this is not a stale graph.
The recorded snapshot is what is wrong
Unserialising coverage.bin.gz from the TIA storage directory (pest --baseline) shows the snapshot itself holds line 134 as executable with an empty test list, while every arm below it has hits:
126 7 tests return true;
132 not executable private function body(...): ?string
134 EXECUTABLE BUT UNCOVERED return match ($type) {
135 9 tests WhatsAppMessageType::Text => ...
136 1 tests WhatsAppMessageType::Image => ...
137 1 tests WhatsAppMessageType::Video => ...
138 1 tests WhatsAppMessageType::Document => ...
139 1 tests WhatsAppMessageType::Reaction => ...
140 1 tests WhatsAppMessageType::Interactive => ...
142 9 tests default => null,
That state cannot arise from real execution: an arm cannot run without the dispatch line running. So the loss happens while recording the snapshot, and the replay is faithfully reporting bad data.
Probe used:
$dir = /* output of `pest --baseline` */;
$coverage = unserialize(gzdecode(file_get_contents($dir.'/coverage.bin.gz')));
$lines = $coverage->getData()->lineCoverage();
// $lines['<file>'][134] === [] → executable, zero hits
Scope
Exactly one line in a 172-file application is affected. For contrast, in the same app and the same snapshot:
- the other 10
return matchlines all record correctly, with 19–58 tests each - one
return match (true)records as not executable, and is correctly not counted against the total
So the classification of a match dispatch line is inconsistent between files, and wherever it lands on "executable" without a recorded hit, the total loses that line permanently.
Ruled out
- Not a stale graph —
--freshimmediately before the failing replay. - Not a missing test — all arms are covered; no test can add a hit the recorder never persisted.
- Not the multi-line
??arm (the one structural difference from the other 10 sites): extracting it into its own method and re-recording produced the identical result on the identical line.
Impact
TIA cannot be combined with a strict coverage threshold: the gate fails on a suite that is genuinely at 100%. The only workaround we found is --no-tia on the coverage command, which removes the speedup exactly where it is largest — here the replayed coverage run takes 4.7s against 116s for the full run.
Environment
- Pest 5.0.3
- PHP 8.5.8 (both
pcovandxdebugloaded; the run setsXDEBUG_MODE=coverage) - Laravel 13
- config:
pest()->tia()->locally();intests/Pest.php - 1093 tests, 172 files under coverage (
<source><include><directory>app</directory>)
Happy to run further probes against this snapshot. The project itself is a client codebase so I cannot share the repo, but I can extract any data you need from the recorded graph or the coverage snapshot.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the two commands, then inspect the TIA snapshot from pest --baseline, especially coverage.bin.gz and getData()->lineCoverage() for the match dispatch line. Trace where TIA records executable coverage and compare that line with its covered arms; done means replayed --coverage --exactly=100.0 reports the same total as the fresh run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100