Guide to Testing with Spring Boot Starter Test

With Spring Boot you only need one dependency to have a solid testing infrastructure: Spring Boot Starter Test. Using this starter, you don't need to manually upgrade testing libraries and keep them compatible. You'll get an opinionated set of libraries and can start writing tests without further setup effort.

This guide gives you a first inside to the swiss-army knife Spring Boot Starter test. This includes an introduction to each testing library added with the starter.

Anatomy of the Spring Boot Starter Test

Every Spring Boot project you create with the Spring Initializr includes the starter for testing:

This starter not only includes Spring specific dependencies and dependencies for auto-configuration, but also a set of libraries for testing. The aforementioned includes JUnit, Mockito, Hamcrest, AssertJ, JSONassert, and JsonPath. They all serve a specific purpose and some can be replaced by each other, which you'll later see.

Nevertheless, this opinionated selection of testing tools is all you need for unit testing. For writing integration tests, you might want to include additional dependencies (e.g. WireMock, Testcontainers or Selenium) depending on your application setup.

With Maven you can inspect all transitive dependencies coming with spring-boot-starter-test using mvn dependency:tree:

If you recently created a Spring Boot application, JUnit 4 is excluded by default (called vintage in JUnit 5).

If your test classes still use JUnit 4, you can remove this exclusion until all your tests are migrated:

While using this starter, you don't need to manually update the versions of all the dependencies. All this is handled by the Spring Boot team and they ensure the different testing dependencies work properly together.

If you need, for some reason, a different version of a dependency coming from this starter, you can override it in your pom.xml:

For now, this is the basic test setup every Spring Boot application uses by default. The following sections cover each test dependency coming with the starter.

Introduction to JUnit

The most important library when it comes to testing is JUnit. It is the major and most used testing framework for Java. This introduction chapter won't cover all features of JUnit and rather focus on the basics.

Before we start with the basics, let's have a short look at the history of JUnit. For a long time, JUnit 4.12 was the main framework version. In 2017 JUnit 5 was launched and is now composed of several modules:

 JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage

The JUnit team invested a lot in this refactoring to now have a more platform-based approach with a comprehensive extension model. Nevertheless, migrating from JUnit 4.X to 5.X requires effort. All annotations, like @Test, now reside in the package org.junit.jupiter.api and some annotations were renamed or dropped and have to be replaced.

A short overview of the differences between both framework versions is the following:

• Assertions reside in org.junit.jupiter.api.Assertions
• Assumptions reside in org.junit.jupiter.api.Assumptions
• @Before and @After no longer exist; use @BeforeEach and @AfterEach instead.
• @BeforeClass and @AfterClass no longer exist; use @BeforeAll and @AfterAll instead.
• @Ignore no longer exists, use @Disabled or one of the other built-in execution conditions instead
• @Category no longer exists, use @Tag instead
• @Rule and @ClassRule no longer exist; superseded by @ExtendWith and @RegisterExtension
• @RunWith no longer exists, superseded by the extension model using @ExtendWith

If your codebase is using JUnit 4, changing the annotations to the JUnit 5 ones is the first step. The most effort is required for migrating custom JUnit 4 rules to JUnit 5 extensions.

For your daily test development, you'll use the following annotations/methods most of the time:

• assertEquals or other assertions to verify the test outcome

• @Test to mark a method as a test

• @BeforeEach/@AfterEach to do setup/teardown tasks before and after a test execution

• @ExtendWith to include an extension like @ExtendWith(SpringExtension.class)

• @ParametrizedTest to run a parameterized test based on different input sources (e.g. CSV file or a list)

For more information on JUnit 5 and migration tips, have a look at the user guide of JUnit 5.

Introduction to Mockito

Mockito is a …

Tasty mocking framework for unit tests in Java

The main reason to use Mockito is to stub methods calls and verify interaction on objects. The first is important if you write unit tests and your test class requires other objects to work.

As your unit test should focus on just testing your class under test, you mock the behavior of the dependent objects of this class.

An example might explain this even better. Let's say we want to write unit tests for the following PricingService:

Our class requires an instance of the ProductVerifier for the method calculatePrice(String productName) to work. While writing a unit test, we don't want to create an instance of ProductVerifier and rather use a stub of this class.

The reason for this is, that our unit test should focus on testing just one class and not multiple together. Furthermore, the ProductVerifier might also need other objects/resources/network/database to properly work, which would result in a test setup hell.

With Mocktio we can easily create a mock (also called stub) of the ProductVerifier. This allows us to 100% control the behavior of this class and make it return whatever we need during a specific test case.

A first test might require the ProductVerifier object to return true. With Mockito this as simple as the following:

The example above should give you the first idea of why we need Mockito.

A second use case for Mockito is to verify an interaction of an object during test execution. Let's enhance the PricingService to report the cheaper price whenever the competitor has the same product in stock:

The notify(String productName) method is void and hence we don't have to mock the return value of this call as it is not used for the execution flow of our implementation. Still, we want to verify that our PricingService actually reports a product.

Therefore, we can now use Mockito to verify that the notify(String productName) method was called with the correct argument. For this to work we also have to mock the ProductReporter during our test execution and can then use Mockito's verify(...) method:

For a more deep-dive introduction to Mockito, consider reading Practical Unit Testing with JUnit and Mockito.

Introduction to Hamcrest

Even though JUnit ships its own assertions within the package org.junit.jupiter.api.Assertions you can still use another assertion library. Hamcrest is such an assertion library.

The assertions you write with Hamcrest follow a more stylized sentence approach which makes it sometimes more human-readable.

While you might write the following assertion with JUnit:

With Hamcrest you do it like this

Besides the fact that it reads more like an English sentence, the order of the parameter is also different. The JUnit assertEquals takes the expected value as a first argument and the actual value as the second argument, Hamcrest does it the other way around:

The Hamcrest Matchers class exposes feature-rich matchers like contains(), isEmpty(), hasSize(), etc. you need for writing tests.

Whether you use JUnit's assertions, Hamcrest or matchers of the assertions library in the next chapter, it depends on your taste. All assertion libraries offer a way to achieve the same, just using a different syntax.

Nevertheless, I would advise you to stick to one library of writing assertions within the same project or at least the same test class.

Introduction to AssertJ

AssertJ is another assertion library that allows you to write fluent assertions for Java tests. It follows a similar approach you already saw with Hamcrest as it makes the assertion more readable.

Let's take our JUnit assertion again as an example for comparison

This would be written with AssertJ like the following:

The available assertions you get are also feature-rich and offer everything you need.

To not get confused during test development, as the Spring Boot Starter includes different libraries to assertions, make sure to import the correct assertion in your test cases.

Mixing them within one assertion is not possible and as they are all named very similar, you should stick to one within the same class file.

Introduction to JSONassert

JSONAssert helps you writing unit tests for JSON data structures. This can be really helpful when testing the API endpoints of your Spring Boot application.

The library works both with JSON provided as String or using the JSONObject / JSONArray class from org.json.

Most of the assertEquals() methods expect a boolean value to define the strictness of the assertion. When it's set to false the assertion won't fail if the JSON contains more fields as expected.

The official recommendation for the strictness is the following:

It is recommended that you leave strictMode off, so your tests will be less brittle. Turn it on if you need to enforce a particular order for arrays, or if you want to ensure that the actual JSON does not have any fields beyond what's expected.

An example helps to understand this:

A JUnit test with the assertion above will be green as the expected field name contains the value duke.
However, if you set the strictness to true, the test above will fail with the following error:

Introduction to JsonPath

Wheres JSONAssert helps you writing assertions for your whole JSON, JsonPath enables you to extract specific parts of your JSON while using a JsonPath expression. The library itself does not provide any assertions and you can use it with any of the assertion libraries already mentioned.

If you are familiar with XPath for XML, JsonPath is like XPath but for JSON.

You can find the whole list of operators and functions you can use with this library on GitHub.

As a first example, let's verify the length of an array and the value of an attribute:

Once you are familiar with the syntax of the JsonPath expression, you can read any property of your nested JSON object:

Summary of the Spring Boot Starter Test

As a summary, these are the key takeaways of this guide: