Guide To Testing With The Spring Boot Starter Test

With Spring Boot, you only need one dependency to have a solid testing infrastructure: Spring Boot Starter Test. Using this starter, you don't need to upgrade testing libraries and keep them compatible manually. You'll get an opinionated set of libraries and can start writing tests without further setup effort.

This guide gives you a first insight into the testing swiss-army knife: Spring Boot Starter Test. This includes an introduction to each testing library that this Spring Boot Starter transitively pulls into your project.

Anatomy of the Spring Boot Starter Test

Every Spring Boot project you create with the Spring Initializr includes the starter for testing:

This starter includes Spring-specific dependencies and dependencies for auto-configuration and a set of testing libraries. This includes JUnit, Mockito, Hamcrest, AssertJ, JSONassert, and JsonPath. These libraries all serve a specific purpose, and some can be replaced by each other, which you'll later see on.

Nevertheless, this opinionated selection of testing tools is all you need for unit testing. For writing integration tests, you might want to include additional dependencies (e.g., WireMock, Testcontainers, or Selenium) depending on your application setup.

With Maven, you can inspect all transitive dependencies coming with spring-boot-starter-test using mvn dependency:tree:

If you recently created a Spring Boot application, JUnit 4 is excluded by default (called vintage in JUnit 5).

If your test classes still use JUnit 4, you can remove this exclusion until all your tests are migrated:

While using this starter, you don't need to update the versions of all the dependencies manually. The Spring Boot team handles all this, and they ensure the different testing dependencies work properly together.

If you need, for some reason, a different version of a dependency coming from this starter, you can override it in your pom.xml:

For now, this is the basic test setup every Spring Boot application uses by default. The following sections cover each test dependency coming with the starter.

Introduction to JUnit

The most important library when it comes to testing is JUnit. It is the major and most used testing framework for Java. This introduction chapter won't cover all features of JUnit and rather focus on the basics.

Before we start with the basics, let's have a short look at the history of JUnit. For a long time, JUnit 4.12 was the main framework version. In 2017 JUnit 5 was launched and is now composed of several modules:

 JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage

The JUnit team invested a lot in this refactoring to have a more platform-based approach with a comprehensive extension model. Nevertheless, migrating from JUnit 4.X to 5.X requires effort. All annotations, like @Test, now reside in the package org.junit.jupiter.api and some annotations were renamed or dropped and have to be replaced.

A short overview of the differences between both framework versions is the following:

• Assertions reside in org.junit.jupiter.api.Assertions
• Assumptions reside in org.junit.jupiter.api.Assumptions
• @Before and @After no longer exist; use @BeforeEach and @AfterEach instead.
• @BeforeClass and @AfterClass no longer exist; use @BeforeAll and @AfterAll instead.
• @Ignore no longer exists, use @Disabled or one of the other built-in execution conditions instead
• @Category no longer exists, use @Tag instead
• @Rule and @ClassRule no longer exist; superseded by @ExtendWith and @RegisterExtension
• @RunWith no longer exists, superseded by the extension model using @ExtendWith

If your codebase uses JUnit 4, changing the annotations to the JUnit 5 ones is the first step. The most effort is required for migrating custom JUnit 4 rules to JUnit 5 extensions.

For your daily test development, you'll use the following annotations/methods most of the time:

• assertEquals or other assertions to verify the test outcome

• @Test to mark a method as a test

• @BeforeEach/@AfterEach to do setup/teardown tasks before and after a test execution

• @ExtendWith to include an extension like @ExtendWith(SpringExtension.class)

• @ParametrizedTest to run a parameterized test based on different input sources (e.g. CSV file or a list)

For more information on JUnit 5 and migration tips, please take a look at the user guide of JUnit 5.

Introduction to Mockito

Mockito is a …

Tasty mocking framework for unit tests in Java

The main reason to use Mockito is to stub methods calls and verify interaction on objects. The first is important if you write unit tests and your class under test has collaborators (other classes that this class depends on).

As your unit test should focus on just testing your class under test, you mock the behavior of any dependent collaborator.

An example might explain this even better. Let's say we want to write unit tests for the following PricingService:

Our class requires an instance of the ProductVerifier for the method calculatePrice(String productName) to work. While writing a unit test, we don't want to create an instance of ProductVerifier and rather use a stub of this class.

This is because our unit test should focus on testing just one class and not multiple together. Furthermore, the ProductVerifier might also need other objects/resources/network/database to work properly, which would result in a test setup hell.

With Mocktio, we can easily create a mock (also called stub) of the ProductVerifier. This allows us to 100% control the behavior of this class and make it return whatever we need during a specific test case.

A first test might require the ProductVerifier object to return true. With Mockito, this as simple as the following:

The example above should give you the first idea of why we need Mockito.

A second use case for Mockito is to verify an interaction of an object during test execution. Let's enhance the PricingService to report the cheaper price whenever the competitor has the same product in stock:

The notify(String productName) method is void Hence, we don't have to mock the return value of this call as it is not used for our implementation's execution flow. Still, we want to verify that our PricingService actually reports a product.

Therefore, we can now use Mockito to verify that the notify(String productName) method was called with the correct argument. For this to work, we also have to mock the ProductReporter during our test execution and can then use Mockito's verify(...) method:

For a more deep-dive introduction to Mockito, consider enrolling in my Hands-On Mocking With Mockito Online Course.

Introduction to Hamcrest

Even though JUnit ships its own assertions within the package, org.junit.jupiter.api.Assertions you can still use another assertion library. Hamcrest is such an assertion library.

The assertions you write with Hamcrest follow a more sentence-like approach which makes it more readable.

While you might write the following assertion with JUnit:

With Hamcrest, you achieve the same with the following assertion:

Besides the fact that it reads more like an English sentence, the parameters' order is also different. The JUnit assertEquals takes the expected value as a first argument and the actual value as the second argument, Hamcrest does it the other way around:

The Hamcrest Matchers class exposes feature-rich matchers like contains(), isEmpty(), hasSize(), etc. you need for writing tests.

Whether you use JUnit's assertions, Hamcrest, or matchers of the assertions library in the next chapter, depends on your personal gusto. All assertion libraries achieve the same – they differ in the syntax and the number of supported assertions.

Nevertheless, I recommend sticking to one assertion library within the same project or at least the same test class.

Introduction to AssertJ

AssertJ is another assertion library that allows you to write fluent assertions for Java tests. It follows a similar approach you already saw with Hamcrest as it makes the assertion more readable.

Let's retake our JUnit assertion as an example for comparison:

This would be written with AssertJ like the following:

The available assertions you get are also feature-rich and offer everything you need.

To not get confused during test development, as the Spring Boot Starter includes different libraries to assertions, make sure to import the correct assertion in your test cases.

Mixing them within one assertion is not possible, and as they are all named very similar, you should stick to one within the test class.

Introduction to JSONassert

JSONAssert helps you writing unit tests for JSON data structures. This can be really helpful when testing the API endpoints of your Spring Boot application.

The library works both with JSON provided as String or using the JSONObject / JSONArray class from org.json.

Most of the assertEquals() methods expect a boolean value to define the strictness of the assertion. When it's set to false the assertion won't fail if the JSON contains more fields as expected.

The official recommendation for the strictness is the following:

It is recommended that you leave strictMode off, so your tests will be less brittle. Turn it on if you need to enforce a particular order for arrays, or if you want to ensure that the actual JSON does not have any fields beyond what's expected.

An example helps to understand this:

A JUnit test with the assertion above will be green as the expected field name contains the value duke.
However, if you set the strictness to true, the test above will fail with the following error:

Introduction to JsonPath

Whereas JSONAssert helps you writing assertions for your whole JSON, JsonPath enables you to extract specific parts of your JSON while using a JsonPath expression. The library itself does not provide any assertions, and you can use it with any of the assertion libraries already mentioned.

If you are familiar with XPath for XML, JsonPath is like XPath but for JSON.

You can find the whole list of operators and functions you can use with this library on GitHub.

As a first example, let's verify the length of an array and the value of an attribute:

Once you are familiar with the syntax of the JsonPath expression, you can read any property of your nested JSON object:

Summary of the Spring Boot Starter Test

As a summary, these are the key takeaways of this guide: