Guide to Testing With the Spring Boot Starter Test

With Spring Boot, you only need one dependency to have a solid testing infrastructure: Spring Boot Starter Test. Using this starter, you don't need to upgrade testing libraries and keep them compatible manually. You'll get an opinionated set of libraries and can start writing tests without further setup effort.

This guide gives you a first insight into the testing swiss-army knife: Spring Boot Starter Test. This includes an introduction to each testing library that this Spring Boot Starter transitively pulls into your project. For unit and integration testing Spring Boot applications in general, take a look at this overview.

Anatomy of the Spring Boot Starter Test

Every Spring Boot project we create with the Spring Initializr includes the starter for testing:

This starter includes Spring-specific dependencies and dependencies for auto-configuration and a set of testing libraries. This includes JUnit, Mockito, Hamcrest, AssertJ, JSONassert, and JsonPath. These libraries all serve a specific purpose, and some can be replaced by each other, which we'll later see on.

Nevertheless, this opinionated selection of testing tools is all we need for unit testing. For writing integration tests, we might want to include additional dependencies (e.g., WireMock, Testcontainers, or Selenium) depending on our application setup.

With Maven, we can inspect all transitive dependencies coming with spring-boot-starter-test using mvn dependency:tree:

Unless some of our tests still use JUnit 4, we can remove the JUnit Vintage engine from the starter. The Vintage engine allows running JUnit 3 and 4 tests alongside JUnit 5

UPDATE: Starting with Spring Boot 2.4,  the JUnit Vintage engine is no longer part of the Spring Boot Starter Test. For projects that are not yet fully migrated to JUnit 5, we have to opt-in for support for previous JUnit versions with the following import:

When using this starter, we don't need to update the versions of all the dependencies manually. The Spring Boot parent POM handles all dependency versions, and the Spring Boot team ensures the different testing dependencies work properly together.

If we need, for some reason, a different version of a dependency coming from this starter, we can override it in our properties section of our pom.xml:

For now, this is the basic test setup every Spring Boot application uses by default. The following sections cover each test dependency coming with this starter.

Introduction to JUnit

The most important library when it comes to testing is JUnit. It is the major and most used testing framework for Java. This introduction chapter won't cover all features of JUnit and rather focus on the basics.

Before we start with the basics, let's have a short look at the history of JUnit. For a long time, JUnit 4.12 was the main framework version. In 2017 JUnit 5 was launched and is now composed of several modules:

 JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage

The JUnit team invested a lot in this refactoring to have a more platform-based approach with a comprehensive extension model. Nevertheless, migrating from JUnit 4.X to 5.X requires effort. All annotations, like @Test, now reside in the package org.junit.jupiter.api , and some annotations were renamed or dropped and have to be replaced.

A short overview of the differences between both framework versions is the following:

• Assertions reside in org.junit.jupiter.api.Assertions
• Assumptions reside in org.junit.jupiter.api.Assumptions
• @Before and @After no longer exist; use @BeforeEach and @AfterEach instead.
• @BeforeClass and @AfterClass no longer exist; use @BeforeAll and @AfterAll instead.
• @Ignore no longer exists, use @Disabled or one of the other built-in execution conditions instead
• @Category no longer exists, use @Tag instead
• @Rule and @ClassRule no longer exist; superseded by @ExtendWith and @RegisterExtension
• @RunWith no longer exists, superseded by the extension model using @ExtendWith

If our codebase uses JUnit 4, changing the annotations to the JUnit 5 ones is the first step. The most effort is required for migrating custom JUnit 4 rules to JUnit 5 extensions.

For our daily test development, we'll use the following annotations/methods most of the time:

• assertEquals or other assertions to verify the test outcome

• @Test to mark a method as a test

• @BeforeEach/@AfterEach to do setup/teardown tasks before and after a test execution

• @ExtendWith to include an extension like @ExtendWith(SpringExtension.class)

• @ParametrizedTest to run a parameterized test based on different input sources (e.g., CSV file or a list)

For more information on JUnit 5 and migration tips, please take a look at the excellent user guide of JUnit 5.

Introduction to Mockito

Mockito is a …

Tasty mocking framework for unit tests in Java

The main reason to use Mockito is to stub methods calls and verify interaction on objects. The first is important if we write unit tests and our class under test has collaborators (other classes that this class depends on).

As our unit test should focus on just testing our class under test, we mock the behavior of any dependent collaborator.

An example might explain this even better. Let's say we want to write unit tests for the following PricingService:

Our class requires an instance of the ProductVerifier for the method calculatePrice(String productName) to work. While writing a unit test, we don't want to create an instance of ProductVerifier and rather use a stub of this class.

This is because our unit test should focus on testing just one class and not multiple together. Furthermore, the ProductVerifier might also need other objects/resources/network/database to work properly, which would result in a test setup hell.

With Mocktio, we can easily create a mock (also called stub) of the ProductVerifier. This allows us to 100% control the behavior of this class and make it return whatever we need during a specific test case.

A first test might require the ProductVerifier object to return true.

With Mockito, this as simple as the following:

The example above should give a first idea of why we need Mockito.

A second use case for Mockito is to verify an interaction of an object during test execution. Let's enhance the PricingService to report the lower price whenever the competitor has the same product in stock:

The notify(String productName) method is void Hence, we don't have to mock the return value of this call as it is not used for our implementation's execution flow. Still, we want to verify that our PricingService actually reports a product.

Therefore, we can now use Mockito to verify that the notify(String productName) method was called with the correct argument. For this to work, we also have to mock the ProductReporter during our test execution and can then use Mockito's verify(...) method:

For a more deep-dive introduction to Mockito, consider enrolling for the Hands-On Mocking With Mockito Online Course.

Introduction to Hamcrest

Even though JUnit ships its own assertions within the package, org.junit.jupiter.api.Assertions we can still use another assertion library. Hamcrest is such an assertion library.

The assertions we write with Hamcrest follow a more sentence-like approach which makes it more readable.

While we might write the following assertion with JUnit:

With Hamcrest, we achieve the same with the following:

Besides the fact that it reads more like an English sentence, the parameters' order is also different. The JUnit assertEquals takes the expected value as the first argument and the actual value as the second argument. Hamcrest does it the other way around:

The Hamcrest Matchers class exposes feature-rich matchers like contains(), isEmpty(), hasSize(), etc. we need for writing tests.

Whether we use JUnit's assertions, Hamcrest, or matchers of the assertions library in the next chapter, depends on our personal gusto. All assertion libraries achieve the same – they differ in the syntax and the number of supported assertions.

Nevertheless, I recommend sticking to one assertion library within the same project or at least the same test class.

Introduction to AssertJ

AssertJ is another assertion library that allows writing fluent assertions for Java tests. It follows a similar approach you already saw with Hamcrest as it makes the assertion more readable.

Let's retake our JUnit assertion as an example for comparison:

This would be written with AssertJ like the following:

The available assertions we get are also feature-rich and offer everything we need.

To not get confused during test development, as the Spring Boot Starter includes different libraries to assertions, make sure to import the correct assertion for the test cases.

Mixing them within one assertion is not possible, and as they are all named very similar, we should stick to one within the test class.

Introduction to JSONassert

JSONAssert helps writing unit tests for JSON data structures. This can be really helpful when testing the API endpoints of our Spring Boot application.

The library works both with JSON provided as String or using the JSONObject / JSONArray class from org.json.

Most of the assertEquals() methods expect a boolean value to define the strictness of the assertion. When it's set to false , the assertion won't fail if the JSON contains more fields as expected.

The official recommendation for the strictness is the following:

It is recommended that you leave strictMode off, so your tests will be less brittle. Turn it on if you need to enforce a particular order for arrays, or if you want to ensure that the actual JSON does not have any fields beyond what's expected.

An example helps to understand this:

A JUnit test with the assertion above will be green as the expected field name contains the value duke.
However, if we set the strictness to true, the test above will fail with the following error:

Introduction to JsonPath

Whereas JSONAssert helps writing assertions for entire JSON documents, JsonPath enables us to extract specific parts of our JSON while using a JsonPath expression. The library itself does not provide any assertions, and we can use it with any of the assertion libraries already mentioned.

What XPath is for XML documents, JsonPath is for JSON payloads. JsonPath defines a set of operators and functions that we can use for our expressions.

As a first example, let's verify the length of an array and the value of an attribute:

Once we are familiar with the syntax of the JsonPath expression, we can read any property of our nested JSON object:

Summary of the Spring Boot Starter Test

As a summary, these are the key takeaways of this guide:

PS: For an overview of the entire Java Testing Landscape, take a look at the 30 Testing Tools And Libraries Every Java Developer Must Know eBook.