Separating Integration and Unit Tests with Gradle

· 8 minute read
Gradle
Source

Gradle is a popular build automation tool that supplies functionality for building Java projects through its Java Plugin. The plugin sets up a structure that most projects can adhere to by separating production code from test code with the introduction of SourceSets, where each SourceSet represents a group of Java source files and miscellaneous resource files that the Java files may interact with. When applying the Java plugin (e.g. apply plugin: 'java'), the project structure is initially configured with two default SourceSets: main and test. The main SourceSet contains the production code that is shipped and deployed, whereas the test SourceSet contains various tests that aren’t deployed but are used to verify the integrity of the production code.

In order to achieve separation of concerns we can separate the different types of tests (e.g. unit, integration, acceptance), allowing us to prioritise unit tests and other verification tasks before integration tests in our task execution graph, thus reducing time spent waiting on all of the tests to fail before determining that one early and quick-to-execute unit test had failed early on in the process. Failing fast is an important concept of the Agile software development methodology as it allows the learning process to begin as early as possible, providing the shortest feedback loop available for the developer to begin fixing a failing test.

A short feedback loop is also important for those practising TDD as the developer will often write a failing unit test beforehand which tests the requirements that the, as-of-yet unwritten, production code should satisfy. As such, the unit test itself will fail immediately and can be run again once a portion of the production code has been written by the developer in order to obtain instant feedback regarding the amount of requirements they have satisfied.

Structuring Your Project

Gradle allows you to write your own plugins that can be shared with all of the modules that exist in your project in order to reduce code duplication. This can be achieved by writing the code you wish to share in your own .gradle file, e.g. integTest.gradle for our integration test plugin. This is referred to as a Script Plugin and allows individual sub-projects to import it if they need the functionality it provides. Typically these plugin files are stored alongside the other Gradle configuration files (e.g. the Gradle Wrapper) under the gradle directory.

In the example below our project has two sub-projects: api and rest. The rest sub-project provides a REST interface to our product’s core api sub-project. Our api is tested with unit tests, however, the rest sub-project includes integration tests in which we mock live HTTP connections that query our REST server, allowing us to make assertions on the data that our interface returns to each query.

sample-project
├── api (1)
│   └── src
│	    ├── main
│	    │   ├── java
│	    │   └── resources
│	    ├── test
│	    │   ├── java
│	    │   └── resources
│	    └── build.gradle
│
├── gradle (2)
│   ├── wrapper (3)
│   │   ├── gradle-wrapper.jar
│   │   └── gradle-wrapper.properties
│   └── integTest.gradle (4)
│
├── rest (1)
│   └── src
│	    ├── integTest (5)
│	    │   ├── java
│	    │   └── resources
│	    ├── main
│	    │   ├── java
│	    │   └── resources
│	    ├── test
│	    │   ├── java
│	    │   └── resources
│	    └── build.gradle
│
└── build.gradle
  1. The sub-projects.
  2. The Gradle configuration directory.
  3. Configuration files for the Gradle Wrapper.
  4. Our integration test Script Plugin.
  5. The directories for our new integTest SourceSet.

Creating the SourceSet

The first thing to add to our integTest.gradle file is configuration for our new SourceSet. As mentioned in the preamble, a SourceSet represents a set of Java source files and miscellaneous resources, the directories for which we need to configure relative to the working directory of the sub-project.

The following code creates a new SourceSet named integTest and configures the directory paths for both its Java source files and resource files:

sourceSets {
    integTest {
        java.srcDir 'src/integTest/java'
        resources.srcDir 'src/integTest/resources'
    }
}

With the SourceSet now in place we must configure its dependencies to ensure that it has access to classes it needs to test, as well as provide it with any of the existing classes from unit tests that it may leverage within the integration tests.

dependencies {
    integTestCompile sourceSets.main.output
    integTestCompile sourceSets.test.output (1)

    integTestCompile configurations.compile
    integTestCompile configurations.testCompile (2)

    integTestRuntime configurations.runtime
    integTestRuntime configurations.testRuntime (3)
}
  1. Provides, at compile-time, the classes produced by the main and test SourceSets, allowing the integration tests to access the production code in main and allowing them to reuse any unit test helper methods in test.
  2. Provides, at compile-time, the dependencies that both main and test require in order to successfully compile.
  3. Provides, at run-time, the dependencies that both main and test require to run.

Integrating with the Task Graph

With our SourceSet now configured it is time to add a task to Gradle’s task execution graph that runs the integration tests. The Java plugin provides a check task by default that runs all verification tasks in the project, including test that should be dependent on our integration tests (which run after all of the quicker verification tasks). It is also important that our integration tests write their reports to a directory that is separate from the unit tests to avoid overwriting any tests that share the same name between both SourceSets.

task integTest(type: Test) {
    group = LifecycleBasePlugin.VERIFICATION_GROUP
    description = 'Runs the integration tests.' (1)

    maxHeapSize = '1024m' (2)

    testClassesDir = sourceSets.integTest.output.classesDir
    classpath = sourceSets.integTest.runtimeClasspath (3)

    binResultsDir = file("$buildDir/integration-test-results/binary/integTest")

    reports { (4)
        html.destination = "$buildDir/reports/integration-test"
        junitXml.destination = "$buildDir/integration-test-results"
    }

    mustRunAfter tasks.test (5)
}

check.dependsOn integTest (6)
  1. The group and description are useful to define as they will be shown to anyone who runs gradle tasks, which provides a detailed list of all runnable tasks within the project.
  2. Integration tests are often memory intensive and could use an increase in the JVM’s maximum heap size.
  3. The task requires us to define where both our classes and classpath reside, which we can find from the SourceSet we defined previously.
  4. As mentioned in the previous paragraph, we must define the directories in which our test reports are written for various formats, e.g. binary, HTML, and XML.
  5. As we want our new task to execute after the unit tests, we can use the mustRunAfter ordering rule to ensure that whenever both of these tasks are in the same execution lifecycle (e.g. when running the check task), the unit tests are ran before the integration tests. See: Ordering Tasks.
  6. Finally we add our newly created integTest task to the verification task lifecycle by making the check task depend upon it.

Achieving Fast Failure

In our task’s definition we ensure that it runs after the test task that exists within the same project. However, our project structure defined two sub-projects for which Gradle will create two individual test tasks. For this reason we must re-evaluate all of the other projects once Gradle has finished the Configuration Phase and modify our integTest task to run after any fast-failing tasks (e.g. test) that were created by other sub-projects.

We can hook into this part of the build using the projectsEvaluated listener which, once triggered, allows us to and iterate over each configured project/sub-project to find fast-failing tasks that should be prioritised before our integTest task within the task execution graph.

gradle.projectsEvaluated {
    def quickTasks = [] (1)

    gradle.rootProject.allprojects.each { project -> (2)
        quickTasks.addAll(project.tasks.findAll { it.name == 'test' }) (3)
        quickTasks.addAll(project.tasks.withType(FindBugs))
        quickTasks.addAll(project.tasks.withType(Pmd)) (4)
    }

    quickTasks.each { task ->
        project.tasks.integTest.mustRunAfter task (5)
    }
}
  1. We begin by defining an empty list that will contain tasks that are quick-to-execute.
  2. Then we iterate over every project that is now configured under the rootProject.
  3. Any tasks named test represent the unit tests and are added to the list.
  4. Tasks whose type are FindBugs or Pmd, two static code analysis tools that are quick to analyse the entire codebase, are also added.
  5. Finally we iterate over every quick-to-execute task and ensure that the integTest task always runs after the task whenever they are in the same build lifecycle.

Applying Our Plugin

Now we have written our plugin we can to apply it to whichever sub-project requires integration tests, at which point Gradle will configure the SourceSet and register the integTest task for us. Applying the plugin is as simple as adding the following line to the build.gradle file of whichever sub-project requires integration test support:

apply from: "$rootDir/gradle/integTest.gradle"

Given the example project structure defined earlier in the article, this would mean adding it to the the sample-project/rest/build.gradle file.

Once applied, running gradle rest:integTest will execute just the integration tests (and any tasks that they depend upon), whereas running gradle rest:check will run any static code analysis tools you chose to configure (e.g. FindBugs, PMD), the unit tests under rest/src/test, and finally the integration tests under rest/src/integTest.