Using Azurite to Test Azure Blob Interactions in Spring Boot

Last Updated:  May 24, 2024 | Published: May 24, 2024

When developing applications that interact with cloud services like Azure Blob Storage, we need to have an efficient testing mechanism in place to ensure reliability and consistency during development and Continuous Integration (CI).

Traditional unit tests are insufficient to validate the integration between our application and the cloud service. On the other hand, running tests against an actual cloud service can be expensive, slow and may lead to unwanted side effects.

This is where a tool like Azurite comes into play. It is an open-source, lightweight emulator provided by Microsoft that simulates Azure Storage API, allowing us to write and test applications without the need for an actual Azure subscription.

In this article, we will explore how we can use Azurite along with Testcontainers to write integration tests for our Spring Boot application that interacts with Azure Blob Storage. The working code referenced in the article can be found in this Github repository.

Sample Spring Boot Application Setup

In order for us to test interactions with Azure Blob Storage, we first need to have an application that interacts with it. Our sample application is going to be a Spring Boot project that exposes a service layer for uploading, retrieving, and deleting files in an Azure Blob Storage container.

Let's start by looking at the Maven project setup and the dependencies required for our application:

With the above dependencies, we are all set to develop our application that connects and interacts with a provisioned Azure Blob Storage container. We will look at how each of these dependencies is used within our application in the upcoming sections.

The remaining dependencies we need to declare are specific to the testing scope, where we will be running our integration tests:

The Spring Boot Starter Test a.k.a. the swiss-army knife for testing gives us the basic testing toolbox as it transitively includes JUnit and other utility libraries, that we will require for writing assertions and running our tests.

And Testcontainers, which allows us to run the Azurite emulator inside a disposable Docker container ensuring an isolated environment for our integration tests.

Azure Spring Boot Configuration

In order to facilitate connection between our application and the Blob Storage container, we need the name of the provisioned containerand its corresponding connection string.

We will store these two properties in our project's application.yml file and make use of @ConfigurationProperties to map the defined values to a POJO, which our application will reference to connect to Azure cloud.

We have also added the @NotBlank annotation, to validate the required properties are available when the application starts. The absence of any one of which, would result in the Spring Application Context failing to start up.

Below is a snippet of our application.yaml file where we will be defining the required properties which will be automatically mapped to the above defined class:

The only thing left for us to do is to define a bean of class BlobContainerClient that is provided in the com.azure:azure-storage-blobdependency we had declared before. This class provides various methods to communicate with the cloud service and will be autowired into our service layer.

By defining a bean of BlobContainerClient in the AzureConfiguration class, we have now successfully given our application the capability to interact with the provisioned Blob Storage container.

Azure Blob Storage Spring Boot Service Layer

With the necessary configurations in place, we can now create a service class that leverages the above declared BlobContainerClientbean as a dependency and exposes methods for interacting with the Azure Blob Storage container.

The service we will create provides the following functionalities:

  • Uploading objects to the Blob Store
  • Retrieving objects from the Blob Store
  • Deleting objects from the Blob Store

With the creation of the above service class, we are done with our sample project setup and can proceed with the main agenda of this article: to test the interactions between our application and Azure Blob Storage.

Azurite Testing Setup with Spring Boot

Prerequisite: Running Docker

The prerequisite for running the Azurite emulator via Testcontainers is, as you've guessed it, an up-and-running Docker instance. We need to ensure this prerequisite is met when running the test suite either locally or when using a CI/CD pipeline.

Running Integration Tests with Maven Failsafe Plugin

We will be using the Maven Failsafe Plugin to run our integration tests during the integration-test phase of the build. The plugin executes test methods in classes whose names end with IT (a convention for integration test classes).

Let's add the plugin to our project's pom.xml file:

With the addition of this plugin, we can execute the command mvn integration-test verify to run the integration tests that we will be writing ahead and verify the reliability of our application. To maintain consistency, I would recommend executing this command in CI/CD pipelines before deploying the application to any environment.

Starting Azurite via Testcontainers

At the time of this writing, the latest version of the Azurite image mcr.microsoft.com/azure-storage/azurite is version 3.29.0. We will be using this version in our test class.

Since Testcontainers does not have a dedicated module for Azurite, we will be using the GenericContainer class, which allows us to start any Docker image.

At first glance, the above code in our test class StorageServiceIT looks a little complex, so let's break down the operations we perform:

  • Start a new instance of the Azurite container using the GenericContainer class, exposing the default port of 10000 for the Blob Service.
  • Construct the connection string for the Azurite container using the mapped port.
  • Create a new Blob Storage container within the Azurite instance using the generated connection string.
  • Dynamically configure Azure configuration properties needed by the application to create the required connection bean using @DynamicPropertySource.

With this setup, our application will use the started Azurite container for all Blob Storage interactions during the execution of our integration test cases, providing an isolated and ephemeral testing environment.

NOTE: The Azurite container created will be automatically destroyed once the test suite is completed, hence we do not need to worry about manual cleanups.

Test Cases: Azure Blob Storage Service

Now that we have configured the Azurite container successfully, we can test the functionalities exposed by StorageService class with appropriate test cases:

In the above test case, we create a random file to upload to the Blob Store and invoke the save() method exposed by our service class. Then, we use the BlobContainerClient to verify if a blob with the same name and content has been created successfully.

Similar to the previous test method, we prepare a random file and upload it to the Blob Store. We then invoke the retrieve() method exposed by our service class and verify if the content of the retrieved blob is the same as the one we uploaded.

Now let's proceed with validating the delete functionality of our service class:

In this test case, we repeat the same step of creating and uploading a random file to the Blob Store. Post completion of which, we invoke the delete() method exposed by our service class with the same name of the saved file. To validate the successful delete operation, we attempt to retrieve the deleted blob with it's corresponding name and assert that appropriate exception is thrown.

By writing these comprehensive test cases, we ensure that our StorageService class functions correctly and interacts with the provisioned cloud service as expected. We can confidently deploy our application to production environments, knowing that it has been thoroughly tested and validated against a simulated Azure Blob Storage service. And the best part? we didn't even open the Azure portal to achieve this.

Pro-tip: Reduce Boilerplate Testcontainer Code

In every test class that directly or indirectly tests a functionality related with the storage layer, we will have to repeat the long testcontainer setup that we saw above.

This process leads to repetitive boilerplate code and increased test suite execution time since a dedicated Azurite container is started and stopped for each test class.

In this section I'll be sharing a solution for these problem statments, one that makes use of the lifecycle callback extension available in JUnit 5 called BeforeAllCallback.

Inside our test folder itself, we will be creating a class AzureBlobStorageInitializer which implements the BeforeAllCallbackinterface and fulfills the responsibility of starting and configuring the Azurite container:

We use an AtomicBoolean flag to track the initial invocation to ensure that the Azurite container is started only once in our test suite.

Now we could annotate our test classes with @ExtendWith(AzureBlobStorageInitializer.class) but I think we can do a better job of improving the readability of this solution and making it “cool”.

We simply create a custom marker annotation @InitializeAzureBlobStorage and meta-annotate it with @ExtendWith(AzureBlobStorageInitializer.class):

All that is left now, is to remove the boilerplate Azurite setup from our test class and leverage our newly created annotation @InitializeAzureBlobStorage:

And just like that, we have successfully eliminated the need for writing boilerplate setup code in each of our test classes and significantly reduced the execution time of our test suite by using a shared Azurite container that is started only once.

Similar to the previous setup, the shared container will be destroyed post execution of our test suite.

Conclusion

Through this article, we have explored how we can create an isolated testing environment with Azurite and Testcontainers to write integration tests for our application that interacts with Azure Blob Storage.

Using this detailed approach, we gain confidence in deployments, enjoy faster feedback cycles, and benefit from a cost-effective testing solution. I hope you would benefit from this approach. Let me know your thoughts or questions in the comment section.

The source code demonstrated throughout this article is available in this Github repository. I would encourage you to explore the codebase.

Joyful testing,

Hardik Singh Behl

(Github | LinkedIn | Twitter)

{"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}
>