Definitive Guide to the JaCoCo Gradle Plugin

As discussed in my article about 100% Code Coverage*, a code coverage tool should provide the means not only to measure code coverage, but also to enforce it. This tutorial shows how to measure and enforce code coverage with JaCoCo and its Gradle plugin, but the concepts are also valid for the JaCoCo Maven plugin.

Example Code

Why JaCoCo?

JaCoCo is currently the most actively maintained and sophisticated code coverage measurement tool for the Java ecosystem.

There’s also Cobertura, but at the time of this writing, the latest commit is from 10 months ago and the build pipeline is failing … signs that the project is not actively maintained.

How Does It Work?

JaCoCo measures code coverage by instrumenting the Java bytecode on-the-fly using a Java Agent. This means that it modifies the class files to create hooks that count if a certain line of code or a certain branch have been executed during a test run.

JaCoCo can be used standalone or integrated within a build tool. In this tutorial, we’re using JaCoCo from within a Gradle build.

Basic Gradle Setup

The basic setup is very straightforward. We simply have to apply the jacoco plugin within our build.gradle:

apply plugin: 'jacoco'

In this tutorial, we’re using JUnit 5 as our testing framework. With the current Gradle version, we still have to tell Gradle to use the new JUnit Platform for running tests:

test {
  useJUnitPlatform()
}

Creating a Binary Coverage Report

Let’s run our Gradle build:

./gradlew build

JaCoCo now automatically creates a file build/jacoco/test.exec which contains the coverage statistics in binary form.

The destination for this file can be configured in the jacocoTestReports closure in build.gradle which is documented on the JaCoCo Gradle Plugin site.

Creating an HTML Coverage Report

Since the binary report is not readable for us, let’s create an HTML report:

./gradlew build jacocoTestReport

When calling the jacocoTestReport task, JaCoCo by default reads the binary report, transforms it into a human-readable HTML version, and puts the result into build/reports/jacoco/test/html/index.html.

Note that the jacocoTestReport task simply does nothing when the test.exec file does not exist. So, we should always run the build or test task first.

The following log output is an indicator that we forgot to run the build or test task:

> Task :tools:jacoco:jacocoTestReport SKIPPED`

We can let this task run automatically with every build by adding it as a finalizer for the build task in build.gradle:

test.finalizedBy jacocoTestReport

Enforcing Code Coverage

The JaCoCo Gradle Plugin allows us to define rules to enforce code coverage. If any of the defined rules fails, the verification will fail.

We can execute the verification by calling:

./gradlew build jacocoTestCoverageVerification

Note that by default, this task is not called by ./gradlew check. To include it, we can add the following to our build.gradle:

check.dependsOn jacocoTestCoverageVerification

Let’s look at how to define verification rules.

Global Coverage Rule

The following configuration will enforce that 100% of the lines are executed during tests:

jacocoTestCoverageVerification {
  violationRules {
    rule {
      limit {
        counter = 'LINE'
        value = 'COVEREDRATIO'
        minimum = 1.0
      }
    }
  }
}

Instead of enforcing line coverage, we can also count other entities and hold them against our coverage threshold:

• LINE: counts the number of lines
• BRANCH: counts the number of execution branches
• CLASS: counts the number of classes
• INSTRUCTION: counts the number of code instructions
• METHOD: counts the number of methods

Also, we can measure these other metrics, aside from the covered ratio:

• COVEREDRATIO: ratio of covered items to uncovered items (i.e. percentage of total items that are covered)
• COVEREDCOUNT: absolute number of covered items
• MISSEDCOUNT: absolute number of items not covered
• MISSEDRATIO: ratio of items not covered
• TOTALCOUNT: total number of items

Excluding Classes and Methods

Instead of defining a rule for the whole codebase, we can also define a local rule for just some classes.

The following rule enforces 100% line coverage on all classes except the excluded ones:

jacocoTestCoverageVerification {
  violationRules {
    rule {
      element = 'CLASS'
        limit {
          counter = 'LINE'
          value = 'COVEREDRATIO'
          minimum = 1.0
      }
      excludes = [
        'io.reflectoring.coverage.part.PartlyCovered',
        'io.reflectoring.coverage.ignored.*',
        'io.reflectoring.coverage.part.NotCovered'
      ]
    }
  }
}

Excludes can either be defined on CLASS level like above, or on METHOD level.

If you want to exclude methods, you have to use their fully qualified signature in the excludes like this:

io.reflectoring.coverage.part.PartlyCovered.partlyCovered(java.lang.String, boolean)

Combining Rules

We can combine a global rule with more specific rules:

violationRules {
    rule {
      element = 'CLASS'
      limit {
        counter = 'LINE'
        value = 'COVEREDRATIO'
        minimum = 1.0
      }
      excludes = [
          'io.reflectoring.coverage.part.PartlyCovered',
          'io.reflectoring.coverage.ignored.*',
          'io.reflectoring.coverage.part.NotCovered'
      ]
    }
    rule {
      element = 'CLASS'
      includes = [
          'io.reflectoring.coverage.part.PartlyCovered'
      ]
      limit {
        counter = 'LINE'
        value = 'COVEREDRATIO'
        minimum = 0.8
      }
    }
  }

The above enforces 100% line coverage except for a few classes and redefines the minimum coverage for the class io.reflectoring.coverage.part.PartlyCovered to 80%.

Note that if we want to define a lower threshold than the global threshold for a certain class, we have to exclude it from the global rule as we did above! Otherwise the global rule will fail if that class does not reach 100% coverage.

Excluding Classes from the HTML Report

The HTML report we created above still contains all classes, even though we have excluded some methods from our coverage rules. We might want to exclude the same classes and methods from the report that we have excluded from our rules.

Here’s how we can exclude certain classes from the report:

jacocoTestReport {
  afterEvaluate {
    classDirectories = files(classDirectories.files.collect {
      fileTree(dir: it, exclude: [
        'io/reflectoring/coverage/ignored/**',
        'io/reflectoring/coverage/part/**'
      ])
    })
  }
}

However, this is a workaround at best. We’re excluding some classes from the classpath of the JaCoCo plugin so that these classes will not be instrumented at all. Also, we can only exclude classes and not methods.

Using a @Generated annotation as described in the next section is a much better solution.

Excluding Classes and Methods From Rules and Reports

If we want to exclude certain classes and methods completely from JaCoCos coverage inspection (i.e. from the rules and the coverage report), there is an easy method using a @Generated annotation.

As of version 0.8.2 JaCoCo completely ignores classes and methods annotated with @Generated. We can just create an annotation called Generated and add it to all the methods and classes we want to exclude. They will be excluded from the report as well as from the rules we define.

At the time of this writing, the JaCoCo Gradle plugin still uses version 0.8.1, so I had to tell it to use the new version in order to make this feature work:

jacoco {
  toolVersion = "0.8.2"
}

Excluding Code Generated By Lombok

A lot of projects use Lombok to get rid of a lot of boilerplate code like getters, setters, or builders.

Lombok reads certain annotations like @Data and @Builder and generates boilerplate methods based on them. This means that the generated code will show up in JaCoCo’s coverage reports and will be evaluated in the rules we defined.

Luckily, JaCoCo honors Lombok’s @Generated annotation by ignoring methods annotated with it. We simply have to tell Lombok to add this annotation by creating a file lombok.config in the main folder of our project with the following content:

lombok.addLombokGeneratedAnnotation = true

Missing Features

In my article about 100% Code Coverage I propose to always enforce 100% code coverage while excluding certain classes and methods that don’t need tests. To exclude those classes and methods from both the rules and the report, the easiest way would be to annotate them with @Generated.

However, this can be a dangerous game. If someone just annotates everything with @Generated, we have 100% enforced code coverage but not a single line of code is actually covered!

Thus, I would very much like to create a coverage report that does not honor the @Generated annotation in order to know the real code coverage.

Also, I would like to be able to use a custom annotation with a different name than @Generated to exclude classes and methods, because our code is not really generated.

Conclusion

This tutorial has shown the main features of the JaCoCo Gradle Plugin, allowing to measure and enforce code coverage.

You can have a look at the example code in my github repository.