missing coverage on aarch64 with -C opt-level=3
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 119k
- Forks
- 16.1k
- PR merge metrics
- PR metrics pending
Description
Note: the way this bug appears in the wild, is that when coverage-testing a large program on aarch64 under opt-level=3 and rustc 1.87, many functions appear to be uncovered even though they are called, and the coverage percentage is low. The low coverage percentage is consistent within a single reproducible build, but small changes in the code can cause the code coverage percentage to dramatically jump as the division of code to codegen units varies
if you are encountering this problem, the easiest workaround is to do your coverage testing using one of -C opt-level=2, -C llvm-args=-aarch64-enable-global-merge=false, or -C link-args=-Wl,-no-gc-sections
On an aarch64/Graviton 2 (m6g.2xlarge) processor, doing the following steps
cat > coverage-test.rs << _EOF_
mod x {
pub(crate) fn foo<T>(t: T) {}
#[inline(never)]
pub(crate) fn zzz() {
foo(4u32);
}
}
#[test]
fn my_test() {
x::zzz();
}
_EOF_
rustc coverage-test.rs -C instrument-coverage -C opt-level=3 --test -C codegen-units=16
rm -f *.profraw # remove any pre-existing coverage files
./coverage-test
echo *.profraw | llvm-profdata merge -f - --binary -o prof.bin
$COV export coverage-test --instr-profile prof.bin --format text
results in a coverage of 33% ("functions":{"count":3,"covered":1,"percent":33.333333333333329}) rather than of 100%, as the lines inside the mod x are "skipped".
Using x86-64, or -C opt-level=2, or -C llvm-args=-aarch64-enable-global-merge=false, or -C link-args=-Wl,-no-gc-sections, or rustc 1.86, there is no problem.
Using nightly, the problem still exists.
Meta
rustc --version --verbose:
rustc 1.87.0 (17067e9ac 2025-05-09)
binary: rustc
commit-hash: 17067e9ac6d7ecb70e50f92c1944e545188d2359
commit-date: 2025-05-09
host: aarch64-unknown-linux-gnu
release: 1.87.0
LLVM version: 20.1.1
Investigation
See https://maskray.me/blog/2021-07-25-comdat-and-section-group
-C instrument-coverage generates section groups that look like this for every coverage counter:
group section [ 14] `.group' [__profc__RINvNtCseYCwLgNLg2S_13coverage_test1x3foomEB4_] contains 3 sections:
[Index] Name
[ 15] __llvm_prf_cnts
[ 16] __llvm_prf_data
[ 17] .rela__llvm_prf_data
The __llvm_prf_data contains metadata that needs to be present in order for coverage to count the lines. The __llvm_prf_cnts contains a counter that is incremented every time the expected code section is hit.
Coverage uses the comdat functionality to ensure that as long as the counter is compiled in, the data area is compiled in as well. The __llvm_prf_data is marked as @llvm.compiler.used to ensure only the linker can gc it.
When using the GlobalMerge pass, it merges the __llvm_prf_cnts into one big section, so the section group looks like this:
group section [ 13] `.group' [__profc__RINvNtCseYCwLgNLg2S_13coverage_test1x3foomEB4_] contains 2 sections:
[Index] Name
[ 14] __llvm_prf_data
[ 15] .rela__llvm_prf_data
As the __llvm_prf_cnts has been moved to one big section, and nothing prevents the __llvm_prf_data from being gc'd.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the coverage-test.rs example on aarch64 with rustc 1.87, -C instrument-coverage, -C opt-level=3, and multiple codegen units. Then inspect the GlobalMerge and section-group behavior described in the investigation. Done means the metadata remains available and the example reports complete coverage without relying on the listed workarounds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100