With Spring Boot you only need one dependency to have a solid testing infrastructure: Spring Boot Starter Test. Using this starter, you don't need to manually upgrade testing libraries and keep them compatible. You'll get an opinionated set of libraries and can start writing tests without further setup effort.
This guide gives you a first inside to the swiss-army knife Spring Boot Starter test. This includes an introduction to each testing library added with the starter.
Anatomy of the Spring Boot Starter Test
Every Spring Boot project you create with the Spring Initializr includes the starter for testing:
1 2 3 4 5 | <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> |
This starter not only includes Spring specific dependencies and dependencies for auto-configuration, but also a set of libraries for testing. The aforementioned includes JUnit, Mockito, Hamcrest, AssertJ, JSONassert, and JsonPath. They all serve a specific purpose and some can be replaced by each other, which you'll later see.
Nevertheless, this opinionated selection of testing tools is all you need for unit testing. For writing integration tests, you might want to include additional dependencies (e.g. WireMock, Testcontainers or Selenium) depending on your application setup.
With Maven you can inspect all transitive dependencies coming with spring-boot-starter-test
using mvn dependency:tree
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | mvn dependency:tree [INFO] +- org.springframework.boot:spring-boot-starter-test:jar:2.2.2.RELEASE:test [INFO] | +- org.springframework.boot:spring-boot-test:jar:2.2.2.RELEASE:test [INFO] | +- org.springframework.boot:spring-boot-test-autoconfigure:jar:2.2.2.RELEASE:test [INFO] | +- com.jayway.jsonpath:json-path:jar:2.4.0:test [INFO] | | \- net.minidev:json-smart:jar:2.3:test [INFO] | | \- net.minidev:accessors-smart:jar:1.2:test [INFO] | | \- org.ow2.asm:asm:jar:5.0.4:test [INFO] | +- jakarta.xml.bind:jakarta.xml.bind-api:jar:2.3.2:compile [INFO] | +- org.junit.jupiter:junit-jupiter:jar:5.5.2:test [INFO] | | +- org.junit.jupiter:junit-jupiter-api:jar:5.5.2:test [INFO] | | | +- org.opentest4j:opentest4j:jar:1.2.0:test [INFO] | | | \- org.junit.platform:junit-platform-commons:jar:1.5.2:test [INFO] | | +- org.junit.jupiter:junit-jupiter-params:jar:5.5.2:test [INFO] | | \- org.junit.jupiter:junit-jupiter-engine:jar:5.5.2:test [INFO] | +- org.junit.vintage:junit-vintage-engine:jar:5.5.2:test [INFO] | | +- org.apiguardian:apiguardian-api:jar:1.1.0:test [INFO] | | +- org.junit.platform:junit-platform-engine:jar:1.5.2:test [INFO] | | \- junit:junit:jar:4.12:test [INFO] | +- org.mockito:mockito-junit-jupiter:jar:3.1.0:test [INFO] | +- org.assertj:assertj-core:jar:3.13.2:test [INFO] | +- org.hamcrest:hamcrest:jar:2.1:test [INFO] | +- org.mockito:mockito-core:jar:3.1.0:test [INFO] | | +- net.bytebuddy:byte-buddy-agent:jar:1.10.4:test [INFO] | | \- org.objenesis:objenesis:jar:2.6:test [INFO] | +- org.skyscreamer:jsonassert:jar:1.5.0:test [INFO] | | \- com.vaadin.external.google:android-json:jar:0.0.20131108.vaadin1:test [INFO] | +- org.springframework:spring-core:jar:5.2.2.RELEASE:compile [INFO] | | \- org.springframework:spring-jcl:jar:5.2.2.RELEASE:compile [INFO] | +- org.springframework:spring-test:jar:5.2.2.RELEASE:test [INFO] | \- org.xmlunit:xmlunit-core:jar:2.6.3:test |
If you recently created a Spring Boot application, JUnit 4 is excluded by default (called vintage in JUnit 5).
If your test classes still use JUnit 4, you can remove this exclusion until all your tests are migrated:
1 2 3 4 5 6 7 8 9 10 11 | <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> <exclusions> <exclusion> <groupId>org.junit.vintage</groupId> <artifactId>junit-vintage-engine</artifactId> </exclusion> </exclusions> </dependency> |
While using this starter, you don't need to manually update the versions of all the dependencies. All this is handled by the Spring Boot team and they ensure the different testing dependencies work properly together.
If you need, for some reason, a different version of a dependency coming from this starter, you can override it in your pom.xml
:
1 2 3 4 5 6 7 8 | <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <properties> <mockito.version>3.1.0</mockito.version> </properties> </project> |
For now, this is the basic test setup every Spring Boot application uses by default. The following sections cover each test dependency coming with the starter.
Introduction to JUnit
The most important library when it comes to testing is JUnit. It is the major and most used testing framework for Java. This introduction chapter won't cover all features of JUnit and rather focus on the basics.
Before we start with the basics, let's have a short look at the history of JUnit. For a long time, JUnit 4.12 was the main framework version. In 2017 JUnit 5 was launched and is now composed of several modules:
JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage
The JUnit team invested a lot in this refactoring to now have a more platform-based approach with a comprehensive extension model. Nevertheless, migrating from JUnit 4.X to 5.X requires effort. All annotations, like @Test
, now reside in the package org.junit.jupiter.api
and some annotations were renamed or dropped and have to be replaced.
A short overview of the differences between both framework versions is the following:
- Assertions reside in
org.junit.jupiter.api.Assertions
- Assumptions reside in
org.junit.jupiter.api.Assumptions
@Before
and@After
no longer exist; use@BeforeEach
and@AfterEach
instead.@BeforeClass
and@AfterClass
no longer exist; use@BeforeAll
and@AfterAll
instead.@Ignore
no longer exists, use@Disabled
or one of the other built-in execution conditions instead@Category
no longer exists, use@Tag
instead@Rule
and@ClassRule
no longer exist; superseded by@ExtendWith
and@RegisterExtension
@RunWith
no longer exists, superseded by the extension model using@ExtendWith
If your codebase is using JUnit 4, changing the annotations to the JUnit 5 ones is the first step. The most effort is required for migrating custom JUnit 4 rules to JUnit 5 extensions.
For your daily test development, you'll use the following annotations/methods most of the time:
assertEquals
or other assertions to verify the test outcome
1 2 3 4 5 6 | import static org.junit.jupiter.api.Assertions.*; assertNotNull(result); assertNotEquals("John", "Duke"); assertThrows(NumberFormatException.class, () -> Integer.valueOf("duke")); assertEquals("Hello World", sampleService.getData()); |
@Test
to mark a method as a test
1 2 3 4 5 6 7 8 | public class FirstTest { @Test public void testOne() { assertEquals(4, "Duke".length()); } } |
@BeforeEach
/@AfterEach
to do setup/teardown tasks before and after a test execution
1 2 3 4 5 6 7 8 9 | @BeforeEach public void setup() { this.testData = prepareTestDataSet(); } @AfterEach public void tearDown() { this.personRepository.deleteAll(); } |
@ExtendWith
to include an extension like@ExtendWith(SpringExtension.class)
1 2 3 4 | @ExtendWith(SpringExtension.class) public class OrderServiceTest { } |
@ParametrizedTest
to run a parameterized test based on different input sources (e.g. CSV file or a list)
For more information on JUnit 5 and migration tips, have a look at the user guide of JUnit 5.
Introduction to Mockito
Mockito is a …
Tasty mocking framework for unit tests in Java
The main reason to use Mockito is to stub methods calls and verify interaction on objects. The first is important if you write unit tests and your test class requires other objects to work.
As your unit test should focus on just testing your class under test, you mock the behavior of the dependent objects of this class.
An example might explain this even better. Let's say we want to write unit tests for the following PricingService
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | public class PricingService { private final ProductVerifier productVerifier; public PricingService(ProductVerifier productVerifier) { this.productVerifier = productVerifier; } public BigDecimal calculatePrice(String productName) { if (productVerifier.isCurrentlyInStockOfCompetitor(productName)) { return new BigDecimal("99.99"); } return new BigDecimal("149.99"); } } |
Our class requires an instance of the ProductVerifier
for the method calculatePrice(String productName)
to work. While writing a unit test, we don't want to create an instance of ProductVerifier
and rather use a stub of this class.
The reason for this is, that our unit test should focus on testing just one class and not multiple together. Furthermore, the ProductVerifier
might also need other objects/resources/network/database to properly work, which would result in a test setup hell.
With Mocktio we can easily create a mock (also called stub) of the ProductVerifier
. This allows us to 100% control the behavior of this class and make it return whatever we need during a specific test case.
A first test might require the ProductVerifier
object to return true
. With Mockito this as simple as the following:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; import org.mockito.Mock; import org.mockito.junit.jupiter.MockitoExtension; import java.math.BigDecimal; import static org.junit.jupiter.api.Assertions.assertEquals; import static org.mockito.Mockito.when; @ExtendWith(MockitoExtension.class) // register the Mockito extension public class PricingServiceTest { @Mock // // Instruct Mockito to mock this object private ProductVerifier mockedProductVerifier; @Test public void shouldReturnCheapPriceWhenProductIsInStockOfCompetitor() { when(mockedProductVerifier.isCurrentlyInStockOfCompetitor("AirPods")) .thenReturn(true); //Specify what boolean value to return PricingService cut = new PricingService(mockedProductVerifier); assertEquals(new BigDecimal("99.99"), cut.calculatePrice("AirPods")); } } |
The example above should give you the first idea of why we need Mockito.
A second use case for Mockito is to verify an interaction of an object during test execution. Let's enhance the PricingService
to report the cheaper price whenever the competitor has the same product in stock:
1 2 3 4 5 6 7 8 | public BigDecimal calculatePrice(String productName) { if (productVerifier.isCurrentlyInStockOfCompetitor(productName)) { priceReporter.notify(productName); return new BigDecimal("99.99"); } return new BigDecimal("149.99"); } |
The notify(String productName)
method is void
and hence we don't have to mock the return value of this call as it is not used for the execution flow of our implementation. Still, we want to verify that our PricingService
actually reports a product.
Therefore, we can now use Mockito to verify that the notify(String productName)
method was called with the correct argument. For this to work we also have to mock the ProductReporter
during our test execution and can then use Mockito's verify(...)
method:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | @ExtendWith(MockitoExtension.class) public class PricingServiceTest { // ... rest like above @Mock private ProductReporter mockedProductReporter; @Test public void shouldReturnCheapPriceWhenProductIsInStockOfCompetitor() { when(mockedProductVerifier.isCurrentlyInStockOfCompetitor("AirPods")).thenReturn(true); PricingService cut = new PricingService(mockedProductVerifier, mockedProductReporter); assertEquals(new BigDecimal("99.99"), cut.calculatePrice("AirPods")); verify(mockedProductReporter).notify("AirPods"); //verify the interaction } } |
For a more deep-dive introduction to Mockito, consider reading Practical Unit Testing with JUnit and Mockito.
Introduction to Hamcrest
Even though JUnit ships its own assertions within the package org.junit.jupiter.api.Assertions
you can still use another assertion library. Hamcrest is such an assertion library.
The assertions you write with Hamcrest follow a more stylized sentence approach which makes it sometimes more human-readable.
While you might write the following assertion with JUnit:
1 | assertEquals(new BigDecimal("99.99"), classUnderTest.calculatePrice("AirPods")); |
With Hamcrest you do it like this
1 2 3 4 | import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.*; assertThat(classUnderTest.calculatePrice("AirPods"), equalTo(new BigDecimal("99.99")); |
Besides the fact that it reads more like an English sentence, the order of the parameter is also different. The JUnit assertEquals
takes the expected value as a first argument and the actual value as the second argument, Hamcrest does it the other way around:
1 2 | assertEquals(expected, actual); assertThat(actual, equalTo(expected)); |
The Hamcrest Matchers
class exposes feature-rich matchers like contains()
, isEmpty()
, hasSize()
, etc. you need for writing tests.
Whether you use JUnit's assertions, Hamcrest or matchers of the assertions library in the next chapter, it depends on your taste. All assertion libraries offer a way to achieve the same, just using a different syntax.
Nevertheless, I would advise you to stick to one library of writing assertions within the same project or at least the same test class.
Introduction to AssertJ
AssertJ is another assertion library that allows you to write fluent assertions for Java tests. It follows a similar approach you already saw with Hamcrest as it makes the assertion more readable.
Let's take our JUnit assertion again as an example for comparison
1 | assertEquals(new BigDecimal("99.99"), classUnderTest.calculatePrice("AirPods")); |
This would be written with AssertJ like the following:
1 2 3 | import static org.assertj.core.api.Assertions.assertThat; assertThat(classUnderTest.calculatePrice("AirPods")).isEqualTo(new BigDecimal("99.99")); |
The available assertions you get are also feature-rich and offer everything you need.
To not get confused during test development, as the Spring Boot Starter includes different libraries to assertions, make sure to import the correct assertion in your test cases.
Mixing them within one assertion is not possible and as they are all named very similar, you should stick to one within the same class file.
Introduction to JSONassert
JSONAssert helps you writing unit tests for JSON data structures. This can be really helpful when testing the API endpoints of your Spring Boot application.
The library works both with JSON provided as String
or using the JSONObject
/ JSONArray
class from org.json
.
Most of the assertEquals()
methods expect a boolean
value to define the strictness of the assertion. When it's set to false
the assertion won't fail if the JSON contains more fields as expected.
The official recommendation for the strictness is the following:
It is recommended that you leave strictMode off, so your tests will be less brittle. Turn it on if you need to enforce a particular order for arrays, or if you want to ensure that the actual JSON does not have any fields beyond what's expected.
An example helps to understand this:
1 2 | String result = "{\"name\": \"duke\", \"age\":\"42\"}"; JSONAssert.assertEquals("{\"name\": \"duke\"}", result, false); |
A JUnit test with the assertion above will be green as the expected field name
contains the value duke
.
However, if you set the strictness to true
, the test above will fail with the following error:
1 2 3 4 5 6 7 | String result = "{\"name\": \"duke\", \"age\":\"42\"}"; JSONAssert.assertEquals("{\"name\": \"duke\"}", result, true); java.lang.AssertionError: Unexpected: age at org.skyscreamer.jsonassert.JSONAssert.assertEquals(JSONAssert.java:417) .... |
Introduction to JsonPath
Wheres JSONAssert helps you writing assertions for your whole JSON, JsonPath enables you to extract specific parts of your JSON while using a JsonPath expression. The library itself does not provide any assertions and you can use it with any of the assertion libraries already mentioned.
If you are familiar with XPath for XML, JsonPath is like XPath but for JSON.
You can find the whole list of operators and functions you can use with this library on GitHub.
As a first example, let's verify the length of an array and the value of an attribute:
1 2 3 4 5 | String result = "{\"age\":\"42\", \"name\": \"duke\", \"tags\":[\"java\", \"jdk\"]}"; // Using JUnit 5 Assertions assertEquals(2, JsonPath.parse(result).read("$.tags.length()", Long.class)); assertEquals("duke", JsonPath.parse(result).read("$.name", String.class)); |
Once you are familiar with the syntax of the JsonPath expression, you can read any property of your nested JSON object:
1 | assertEquals("your value", JsonPath.parse(result).read("$.my.nested.values[0].name", String.class)); |
Summary of the Spring Boot Starter Test
As a summary, these are the key takeaways of this guide:
- JUnit is the testing framework to launch tests on the JVM
- Mockito helps you mocking objects and verifying interactions on them
- Pick one assertion library for writing tests: JUnit's built-in assertions, Hamcrest or AssertJ
- Both JSONassert and JsonPath help you writing tests for JSON data structures
For some integration tests, consider including further libraries e.g. WireMock, Testcontainers or Selenium.
Have fun testing your Spring Boot application,
Phil
[…] >> Guide to Testing with Spring Boot Starter Test [rieckpil.de] […]
[…] it comes to testing the application, the swiss army knife Spring Boot Starter Test already provides all test dependencies we […]
[…] the Spring Boot Starter Test serves multiple […]
[…] retrieves all todos from the database. Testing is done with JUnit 5 (Jupiter) that is part of the Spring Boot Starter Test dependency (aka. swiss-army knife for testing Spring […]
[…] using MockMvc. Including both the Spring Boot Starter Web and the Spring Boot Starter Test (aka. swiss-army for testing Spring Boot applications) is everything you […]
[…] those of you that use Spring Boot and the Spring Boot Starter Test, you can update to Spring Boot Version 2.4.0-M2. This version includes the Mocktio dependency in a […]