bazelbuild / bazelbuild/bazel

C++ Coverage on MacOS is completely broken

Open
#10,457 8 comments 0 reactions 0 assignees View on GitHub
P3 team-Rules-CPP type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

Coverage with gcov:

1. GCOV_PREFIX_STRIP is set incorrectly
With a value of 3, the paths end up like this:
bazel-testlogs/cpp/foo_test/_coverage/_bazel_ulfjack/312a82d9f5d7857ee9e5303755329142/sandbox/darwin-sandbox/4/execroot/__main__/bazel-out/darwin-fastbuild/bin/cpp/_objs/foo_test/main.gcda

The collect-coverage.sh script doesn't allow overriding GCOV_PREFIX_STRIP. I have a patch to not set it if it's already set in the test env. On this machine, I have to set it to 10.

2. The latest gcov does not support the -i flag
The -i flag is hard-coded in collect_cc_coverage.sh with no override possible.

3. The gcov log uses lower-case "creating"
The collect_cc_coverage.sh only checks for upper-case "Creating".

4. The generated output is not in intermediate gcov format as expected by lcov_merger.

Coverage with llvm-profdata (--features=llvm_coverage_map_format):

1. COVERAGE_GCOV_PATH is set to gcov
This is obviously wrong. It needs to be set to llvm-profdata. However, llvm-profdata is not on the path, and there does not seem to be a way to override the location of gcov. I tried setting GCOV=/path/to/llvm-profdata before calling Bazel, but that had no effect.

2. The lcov_merger simply copies the .profdata file to coverage.dat
At least this allows users to post-process manually, but it's still unfortunate.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the macOS coverage flow and reading collect-coverage.sh, collect_cc_coverage.sh, and lcov_merger. Check how GCOV_PREFIX_STRIP, gcov options, coverage-tool paths, and generated formats are selected for both gcov and llvm-profdata. Done means the reported macOS coverage cases work without breaking existing coverage behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
build-system, testing
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.