jacoco / jacoco/jacoco

Wrong coverage rate due to loss of precision

Open
#449 13 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

component: report type: enhancement
Dominant language
Java
Stars
4.6k
Forks
1.2k
Avg merge
1d 9h
Merged PRs (30d)
23

Description

JaCoCo reports wrong coverage rate due to loss of precision.

Steps to reproduce

JaCoCo version: 0.7.7.201606060606
Operating system: Linux 4.4.0-21-generic #37-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux
Tool integration: Maven
maven-jacoco-plugin configuration: jacoco-maven-plugin-config.txt

Expected behaviour

I assume that JaCoCo uses the following formulas to calculate final coverage rates:

instructions coverage rate = ((total instructions count - missed instructions count) / total instructions count) * 100%
branches coverage rate = ((total branches count - missed branches count) / total branches count) * 100%

For example in JaCoCo coverage report I have the following numbers:

total instructions count = 60501
missed instructions count = 506

and

total branches count = 7652
missed branches count = 78

So, the instructions coverage rate and branches coverage rate will be:

instructions coverage rate = ((60501 - 506) / 60501) * 100% = 0.991636501 * 100% = 99.1636501...%
branches coverage rate = ((7652 - 78) / 7652) * 100% = 0.989806586 * 100% = 98.9806586...%

Actual behaviour

When I execute mvn clean verify , JaCoCo fails the build due to the fact that coverage rate for branches is 0.98 which is lower than the specified threshold (see jacoco-maven-plugin-config.txt):

  • for instructions: 0.99
  • for branches: 0.99

However, in the coverage report I see that:
instructions coverage rate = 99%
branches coverage rate = 99%

Problem description

There is a wrong coverage rate in the report due to loss of precision.
jacoco:check phase uses correct values which are calculated with the help of formulas specified above. As a result it is unclear for user to determine where is a problem: reports look OK, but jacoco:check fails the build.

Proposal

Use equal precision in both report coverage rates and check phase.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the discrepancy with the attached jacoco-maven-plugin configuration and mvn clean verify, comparing the report's displayed rates with the jacoco:check result. Trace where report percentages and check thresholds are calculated or formatted; done means both use equal precision and the branch threshold behaves consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.