Nothing is more tiresome than long feedback cycles due to slow builds. I've recently invested one day to improve the integration test setup for a larger project. The result was impressive: the build time (running mvn clean verify) went down from 25 minutes to 5 minutes by solely focussing on the test setup. All of this was possible by making the most of the Spring Test Context Caching mechanism. This blog post will introduce you to the context caching mechanism, including my recommendations to make the most of this feature to reduce build times drastically.
Spring Test Context Caching Mechanism
Whenever your tests go beyond trivial unit testing, and you include Spring Test features, your tests require a running Spring Context (e.g. @SpringBootTest, @WebMvcTest, @DataJpaTest). Spring's excellent test support in combination with Spring Boot allows creating a sliced application context that fits your testing purpose (e.g., include only web-layer related beans when using @WebMvcTest).
Starting a Spring Context for your test takes a reasonable amount of time. Mainly if you populate the whole context using @SpringBootTest. This adds overhead (time-wise) to your test execution and may drastically increase your total build time if each test starts its context.
Fortunately, Spring Test provides a mechanism to cache an already started application context and reuse it for subsequent tests with similar context requirements.
Internally, Spring uses a Map to store the different application contexts. As not every application context can be used for all tests (imagine using a web application context for a persistence-layer test), Spring Test creates a key to uniquely identify the context setup based on various properties/configuration parameters:
locations(from@ContextConfiguration)classes(part of@ContextConfiguration)contextInitializerClasses(from@ContextConfiguration)contextCustomizers(fromContextCustomizerFactory) – e.g.@DynamicPropertySource,@MockBeanand@SpyBean.contextLoader(part of@ContextConfiguration)parent(from@ContextHierarchy)activeProfiles(coming from@ActiveProfiles)propertySourceLocations(from@TestPropertySource)propertySourceProperties(from@TestPropertySource)resourceBasePath(part of@WebAppConfiguration)
Whenever one of these configuration parameters changes, the context key will also change. If there is no entry for a test context that Spring is about to start in the Map (a cache miss), Spring Test starts a new context. For every cache hit, Spring Test will reuse the context and won't start a new one.
If most of your tests share the same context configuration, you can reuse the Spring context for multiple tests and reduce the overall build times.
That's the theory. Let's take a look at what usually goes wrong or prevents this neat feature of Spring Test from shining.
Pitfalls When Trying to Reuse a Context
Most of the time, the usage of either @MockBean or @SpyBean prevents you from reusing an already started application context. What I've seen a lot is using these annotations to short-circuit the testing functionality of your application.
Consider your application subscribes to a JMS queue, and you want to verify the processing. During your integration test, you put a message into your queue (e.g., while using Testcontainers) and start the whole Spring context (@SpringBootTest). The lazy developer might add mocked beans for the processing logic and will just verify that a consumer processed the message.
Some developers might argue that testing everything together is hard, and they simply don't know how to do it. While this is a valid argument, consider not to start the whole Spring context or ask yourself if you are actually testing library functionality.
A different place for this short-circuiting can be found when testing the web layer with @SpringBootTest:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | @SpringBootTest(webEnvironment = RANDOM_PORT) public class SecondApplicationTest { @Autowired private TestRestTemplate testRestTemplate; @MockBean private PersonService personService; @Test public void testPublicEndpoint() { when(personService.getPerson()).thenReturn("testPerson"); String result = this.testRestTemplate .getForObject("/", String.class); assertEquals("testPerson", result); } } |
For the test case above, MockMvc and @WebMvcTest might be sufficient.
Another pitfall I often see is the excessive use of @DirtiesContext. There might be good reasons to use this annotation, e.g., when you modify the global application state in a test. But most of the time, it's rather a laziness of the developers when they encounter an issue while running all tests together. Usually, this occurs when testing messaging or database-related parts.
Rather than taking the time to understand the root cause of the problem, you can easily help yourself with @DirtiesContext. Using this annotation, you mark the Spring context as dirty and Spring Test won't reuse it. Once someone introduces this annotation, the chances are high that the next developer will pick it up, and nobody will take time to fix it as the window is already broken.
Furthermore, using multiple different profiles for your tests with @ActiveProfiles is also a common pitfall. Try to avoid having multiple test profiles (e.g. it, it-web-test, it-database) and limit it to the smallest amount possible.
Let's take a look at further techniques to make the most of the context reusability feature of Spring Test.
Techniques for Efficient Context Reusing
In general, you should avoid modifying any global state inside your application context that prevents reusing it.
Make sure to always clean up resources and leave the Spring Context clean after each test execution.
Try to stick as much as possible to the same context configuration for your integration tests. To achieve this, you can either introduce an abstract parent class that includes your common configuration:
| 1 2 3 4 5 6 7 8 9 10 | @AutoConfigureMockMvc @ContextConfiguration(initializers = CustomInitializer.class) @SpringBootTest(webEnvironment = RANDOM_PORT) public abstract class AbstractIntegrationTest { @BeforeAll public static void commonSetup() { // e.g. provide WireMock stubs } } |
… or write a custom annotation that includes all configuration annotations, e.g. @FullIntegrationTest.
Whenever you are tempted to short-circuit a test using @MockBean, rethink. Especially when using this in conjunction with @SpringBootTest. There is a good chance that Spring provides an annotation to start a sliced context for the part of your application you want to test. Furthermore, you can also prepare a custom context with only the beans you need.
If you currently parallelize the execution of, e.g., your integration tests using either JUnit 5 or the Failsafe plugin, Spring Test might not be able to reuse the contexts. If your tests don't always run within the same process, you won't benefit from the context caching mechanism. Try disabling the parallelization and see if the context caching outperforms the parallelization of tests.
Insights Into the Context Caching of Spring Test
If you are curious why your current test does not reuse an already started context, you can activate debug logs:
| 1 | logging.level.org.springframework.test.context.cache=DEBUG |
This will also show statistics about the context cache:
| 1 2 | ... Storing ApplicationContext [396485834] in cache under key [[WebMergedContextConfiguration@222a59e6 testClass = ApplicationTests, locations = '{}', classes = '{class de.rieckpil.blog.Application}', contextInitializerClasses = '[]', activeProfiles = '{}', propertySourceLocations = '{}', ... ]] |
The default cache size is 32, but you can quickly increase or decrease this value:
| 1 | spring.test.context.cache.maxSize=42 |
Whenever the context cache limit is reached, Spring Test uses a least recently used (LRU) eviction policy to make space for the next context.
PS: I even gave a talk about this Spring feature at the Spring I/O 2022 in Barcelona:
If you want to learn more about this feature of Spring Test in action and testing Spring Boot applications in general, consider enrolling for the Testing Spring Boot Applications Masterclass.
You can find the demo application with further test examples on GitHub.
Happy context caching with Spring Test,
Phil
