Testcontainers is now for almost a year part of my core testing libraries set. It allows you to control Docker containers for external parts of your application (e.g. database or messaging queue). This helps a lot when you write integration tests. The integration of Testcontainers with Spring Boot and JUnit is excellent and they improve it even more with every release. When you use Testcontainers for your integration test, you'll recognize some delays when you launch the test. With the 1.12.3 release, they introduced a new feature (in alpha stage) to reuse existing containers. Let's see how we can use this new feature to reduce the time spent on integration tests for shorter build times.
Update: You can now find a visual demo of this Testcontainers feature on YoutTube.
Spring Boot Testcontainers Maven Project Setup
To demonstrate reusable containers with Testcontainers, we're using a sample Spring Boot application that connects to a PostgreSQL database.
The application exposes one endpoint which retrieves all todos from the database. Testing is done with JUnit 5 (Jupiter) that is part of the Spring Boot Starter Test dependency (aka. swiss-army knife for testing Spring applications):
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.6.0</version> <relativePath/> <!-- lookup parent from repository --> </parent> <groupId>de.rieckpil.blog</groupId> <artifactId>testcontainers-reuse-existing-containers</artifactId> <version>0.0.1-SNAPSHOT</version> <name>testcontainers-reuse-existing-containers</name> <description>Reuse existing Containers with Testcontainers</description> <properties> <java.version>11</java.version> <testcontainers.version>1.16.2</testcontainers.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-jpa</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.flywaydb</groupId> <artifactId>flyway-core</artifactId> </dependency> <dependency> <groupId>org.postgresql</groupId> <artifactId>postgresql</artifactId> <scope>runtime</scope> </dependency> <dependency> <groupId>org.testcontainers</groupId> <artifactId>postgresql</artifactId> <version>${testcontainers.version}</version> <scope>test</scope> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> <!-- build section --> </project> |
Furthermore, we're using the Maven Failsafe plugin to run the integration tests separated from our unit tests:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 | <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-failsafe-plugin</artifactId> <version>3.0.0-M5</version> <executions> <execution> <goals> <goal>integration-test</goal> <goal>verify</goal> </goals> </execution> </executions> </plugin> |
By following the default convention, our integration tests need the postfix IT and the Failsafe plugin will pick them up automatically.
Requirements to Reuse Containers with Testcontainers
To enable the reuse of containers with Testcontainers, we have the following requirements:
- use
.withReuse(true)for our container definition - opt-in for reusable containers inside our
~/.testcontainers.propertiesfile - use a manual container lifecycle control: singleton containers or respectively start the containers manually with e.g.
@BeforeAll(it won't work with the JUnit 4 rule or JUnit Jupiter extension@Testcontainers)
For the first requirement, we adjust our container definition and add .withReuse(true). This method is part Testcontainers GenericContainer class:
| 1 2 3 4 5 6 | static PostgreSQLContainer<?> postgreSQLContainer = new PostgreSQLContainer<>("postgres:10-alpine") .withDatabaseName("test") .withUsername("duke") .withPassword("s3cret") .withReuse(true); |
For the second requirement, we modify the Testcontainers configuration file within our home directory. If we've never run a test with Testcontainers before, we can create this file manually:
| 1 2 3 4 5 | cat ~/.testcontainers.properties #Modified by Testcontainers #Thu Jun 18 07:46:35 CEST 2020 testcontainers.reuse.enable=true docker.client.strategy=org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy |
But the last requirement is where people often wonder why they can't reuse the container as they either use the JUnit 4 rule or the JUnit 5 extension. But both the rule and the extension (enabled with @Testcontainers) tear down all containers after the test execution.
Reusing Containers for Integration Tests
Let's start using this feature by writing the first integration tests.
The tests are a basic example of testing the Spring Boot application as a whole. We're using Testcontainers to start the mandatory PostgreSQL database and verify that the endpoint is returning data.
As the setup for the different tests is similar, we're using an abstract class to define the basic configuration for our integration tests:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | @SpringBootTest(webEnvironment = RANDOM_PORT) public abstract class BaseIT { static final PostgreSQLContainer<?> postgreSQLContainer; static { postgreSQLContainer = new PostgreSQLContainer<>(DockerImageName.parse("postgres:13")) .withDatabaseName("test") .withUsername("duke") .withPassword("s3cret") .withReuse(true); postgreSQLContainer.start(); } @DynamicPropertySource static void datasourceConfig(DynamicPropertyRegistry registry) { registry.add("spring.datasource.url", postgreSQLContainer::getJdbcUrl); registry.add("spring.datasource.password", postgreSQLContainer::getPassword); registry.add("spring.datasource.username", postgreSQLContainer::getUsername); } } |
With .withReuse(true) we're telling Testcontainers to reuse the container and not shut it down after a test.
Furthermore, we're using a singleton container and injecting the new data source URL, password, and username using @DynamicPropertySource. This is available since Spring Boot 2.2.6. For Spring Boot versions before 2.2.6 and/or JUnit 4, take a look at this blog post that explains the Testcontainers setup for different Spring Boot and JUnit versions.
A first integration test can now extend the BaseIT and verify different parts of the application:
| 1 2 3 4 5 6 7 8 9 10 11 12 | class ApplicationIT extends BaseIT{ @Autowired private TestRestTemplate testRestTemplate; @Test void contextLoads() { ResponseEntity<JsonNode> result = testRestTemplate.getForEntity("/todos", JsonNode.class); assertEquals(200, result.getStatusCodeValue()); } } |
A second test can also reuse the common test configuration:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | @SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT) class SecondApplicationIT extends BaseIT{ @Autowired private TestRestTemplate testRestTemplate; @Autowired private TodoRepository todoRepository; @AfterEach public void cleanup() { this.todoRepository.deleteAll(); } @Test void contextLoads() { this.todoRepository.saveAll(List.of(new Todo("Write blog post", LocalDateTime.now().plusDays(2)), new Todo("Clean appartment", LocalDateTime.now().plusDays(4)))); ResponseEntity<ArrayNode> result = this.testRestTemplate.getForEntity("/todos", ArrayNode.class); assertEquals(200, result.getStatusCodeValue()); assertTrue(result.getBody().isArray()); assertEquals(2, result.getBody().size()); } } |
While these two tests would also share the same container (without the reuse feature) as the base class only starts the container when it's loaded, we still have a big benefit now. After the test execution, Testcontainers won't stop the container and still keep it running. This allows subsequent tests to use the already running container and save initial time for container setup.
Another important thing to note is that our container configuration has to match a previous configuration, as we won't be able to reuse an existing container otherwise.
Let's say we use PostgreSQL 13 but define two different database names and one integration test requires PostgreSQL 10 (for whatever reasons):
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | // definition in one test static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer<>("postgres:10-alpine") .withDatabaseName("test") .withUsername("duke") .withPassword("s3cret") .withReuse(true); // definition in another test static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer<>("postgres:13") .withDatabaseName("differentDatabaseName") .withUsername("duke") .withPassword("s3cret") .withReuse(true); // definition in the last test static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer<>("postgres:13") .withDatabaseName("test") .withUsername("duke") .withPassword("s3cret") .withReuse(true); |
Testcontainers will then start the different containers as they can't be reused (which makes total sense).
The result after executing the test is the following:
| 1 2 3 | b0df4733babb postgres:13 "docker-entrypoint.s…" 19 seconds ago Up 18 seconds 0.0.0.0:32778->5432/tcp inspiring_dewdney f55ba18bddc5 postgres:10-alpine "docker-entrypoint.s…" 12 minutes ago Up 12 minutes 0.0.0.0:32775->5432/tcp stupefied_tu 9add8969fae1 postgres:13 "docker-entrypoint.s…" 23 minutes ago Up 23 minutes 0.0.0.0:32769->5432/tcp affectionate_curran |
They are all up and running after our test. If we execute all our tests again, they will be significantly faster as Testcontainers can reuse the containers from our last test execution.
As a reference, on my machine, the three integration tests from above take 20 seconds running with a cold start and only take 10 seconds when reusing containers.
Summary of Reusing Containers
Even though this feature is still in alpha state, it works great for basic usage of Testcontainers. Especially for local development and if you practice TDD (Test Driven Development), this sparks joy as your feedback cycle is now really short.
If your setup allows you to reuse existing containers with Testcontainers (e.g., database is cleaned after each test), this feature can save you a lot of time. Also, you should ensure that the running containers can handle multiple connections as Spring Test provides a context caching mechanism. This allows you to spawn multiple application contexts during your test execution, all connecting to the same container (if you configure it in such a way).
The only thing you have to keep in mind is that the containers are still running after your test execution. If you work on several projects throughout the day you might want to shut them down (e.g. docker rm $(docker ps -a -q)) to avoid conflicts or save some resources on your machine.
You can find the sample application on GitHub with instructions on how to run it on your machine.
Searching for more content on Testcontainers and writing efficient integration tests?
- Write Spring Boot integration tests with Testcontainers (JUnit 4 & 5)
- Test Spring applications using AWS with Testcontainers and LocalStack
- Spring Boot Functional Tests with Selenium and Testcontainers
Have fun reusing your containers with Testcontaniners to accelerate your build times,
Phil
Hi! It’s now possible to easily reuse containers: https://www.testcontainers.org/test_framework_integration/manual_lifecycle_control/
Hi Martin,
yes, I’m using the Singleton Container approach from the link you referenced.
Great Job Sr.
It really works for me.
Thank you very much for your feedback, Daniel!
Lifesaver! thank you!
You’re welcome, I’m glad the article helped you 🙂
Amazing tips, thank you! When also using
.withLabel("reuse.ID", ...), I can run, locally, integration tests for different release branches in parallel, whether they’re using the same or different DB versions. Love it!Any advice about how to reuse testcontainers in Azure CI builds? Where would one place the
.testcontainers.propertiesfile?Hey Boris,
I’m glad you like the article. You can store this file at various locations, see all possible options here.
Kind regards,
Philip