llvm / llvm/llvm-project

[Coverage] Line coverage contains invalid counts

Open
#171,822 3 comments 0 reactions 0 assignees View on GitHub
coverage
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

In our codebase, `llvm-cov export --format=lcov` produces some weird, unrealistic high line hit counts, especially considering the line counts surrounding them within the same region. Find an example of a slightly obfuscated function together with the LCOV output below:

```cpp
int DisplayIdManager::findNewDisplayID( const Framework &workingFw, const Framework &orderingFw,
const QString &businessKey, const QString &orderKey )
{ // line 330
Q_ASSERT( !parentBusinessKey.isEmpty() && !orderKey.isEmpty() );

// try to find in cache at first
int displayID = displayIdCache.maxDisplayId( orderKey, businessKey );
if ( displayID > 0 )
{
++displayID;
displayIdCache.setMaxDisplayId( orderKey, businessKey, displayID );
return displayID;
}

int workingVersionID = getVersionIdByType( workingFw, versionTypes_c::WORKING ); // line 342
int orderingVersionID = getVersionIdByType( orderingFw, versionTypes_c::ORDERING );

// unrelated comment
const auto &foo = fetch< SomeDataList >( workingFw );
...
```

```lcov
SF:/opt/.../display_id_mgr.cpp
...
DA:330,77568
DA:331,77568
DA:334,77568
DA:335,77568
DA:336,78033
DA:337,78033
DA:338,78033
DA:339,78033
DA:340,78033
DA:342,18446744073709551151
DA:343,18446744073709551151
DA:346,18446744073709551151
DA:348,18446744073709551151
DA:349,2
DA:350,2
...
```

It is weird, because these lines are unconditionally executed and the count of lines before and after seems normal.

In total, it is 38 from 1.5M analyzed lines that have this issue.

The issue appears when trying to upload the LCOV file to a service like e.g. CodeScene. Their CLI tools expect a 32-bit integer. I couldn't find a strict specification of the LCOV format. So perhaps this is not an issue here but with the CodeScene CLI.

LLVM version: 16.0.2

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the invalid DA counts with llvm-cov export --format=lcov using LLVM 16.0.2 and the obfuscated function pattern shown. Trace whether the oversized values are produced by llvm-cov or arise from LCOV interpretation; done means identifying the responsible side and confirming the counts are valid for the affected cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
devtools, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.