Tia: coverage cache downloaded from another machine empties the coverage report — no per-file rows, bogus Total: 100.0%, --min passes
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 11.7k
- Forks
- 538
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 8
Description
Summary
When a TIA baseline recorded on one machine is downloaded and replayed on another (pest()->tia()->baselined()), the coverage cache (coverage.bin.gz) shipped inside the baseline artifact carries the recording machine's absolute paths. After CoverageMerger merges it with the local run's coverage, the final report contains two path roots. The resulting report has zero file nodes, so Pest prints an empty per-file table and Total: 100.0 % — and --coverage --min=X passes regardless of the real coverage. The check fails open, with no warning.
Environment
- pest
v5.0.1, phpunit13.2.4, phpunit/php-code-coverage14.2.3 - Recording machine: GitHub Actions
ubuntu-latest(project at/home/runner/work/<repo>/<repo>) - Consuming machine: macOS, PHP 8.4.13, xdebug 3.4.7 (
XDEBUG_MODE=coverage), project at/Users/<user>/...
Reproduction
- On CI, record a baseline with coverage and publish it via the documented flow:
run: ./vendor/bin/pest --parallel --tia --coverage --fresh # upload the directory printed by `pest --baseline` as the pest-tia-baseline artifact - On a developer machine (different absolute project path), with
pest()->tia()->baselined()intests/Pest.php:
The baseline (includingXDEBUG_MODE=coverage ./vendor/bin/pest --parallel --tia --coverage --min=68.9coverage.bin.gz) is downloaded, tests replay, and the output is:
No per-file rows, bogus 100.0 % total,Tests: 1 skipped, 2619 passed (8738 assertions, 14 uncached, 2606 replayed) ──────────────────────────────────────────────────────────── Total: 100.0 %--minsatisfied. Exit code 0.
What we observed inside the merged cache
Unserializing the local coverage.bin.gz after one replay:
lineCoverage()contained 2558 files: 1285 under the local root (/Users/...) and 1273 under the recording runner's root (/home/runner/work/...);getReport()on that object produced 0Filenodes (631Directorynodes),numberOfExecutableLines() === 0, and an empty total percentage;- after dropping the foreign-root entries from the data and rebuilding, the report came back to life: 1274
Filenodes, 26 061 executable lines, a real total.
The empty report follows from SebastianBergmann\CodeCoverage\Node\Builder::addItems(): it reconstructs each filename from the common path of the whole data set and silently skips any file whose reconstructed path fails is_file(). With two disjoint roots in the data, every file is skipped. Pest's Coverage::report() then iterates zero File nodes (empty table) and percentageOfExecutedLines() over 0/0 renders as 100.0 %, which --min happily accepts.
Root cause
The serialized CodeCoverage cache is machine-specific (absolute paths baked into the data), but:
BaselineSyncships it across machines — the downloaded artifact'scoverage.bin.gzis written verbatim into local state (BaselineSync::download()→state->write(Tia::KEY_COVERAGE_CACHE, $payload['coverage']));CoverageMerger::applyIfMarked()merges it with the local report without rebasing the cached paths to the local project root.
Suggested direction
Any of these would fix the failure mode, in decreasing order of usefulness:
- Rebase the cached coverage paths to the consumer's project root before merging (store the recording root alongside the cache, or derive it from the graph);
- Don't ship
coverage.bin.gzin the baseline artifact / don't import it on fetch — let each machine build its own coverage baseline; - At minimum, detect a foreign root in the cache and discard it with a warning instead of producing an empty report that passes
--minat a fake 100.0 %.
Related: #1792 fixes a separate defect in the same merge path (stale cached entries of changed files deflating merged totals). The two compose: cross-machine caches fail open (this issue), same-machine stale caches fail closed (#1792).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with BaselineSync::download() and CoverageMerger::applyIfMarked(), then trace how the downloaded coverage.bin.gz paths reach Coverage::report(). Reproduce the documented TIA baseline flow with different project roots and verify that a foreign-root cache no longer produces an empty report or a false 100.0% result that satisfies --min.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100