RESTful communication is the de-facto standard for interchanging data in a microservice-based environment. Usually, every participating microservice offers different parts of the application's domain in a RESTful way and calls other microservices to gather data for e.g. a different part of the application's domain. Resilience and efficient data interchange is, therefore, a key factor when it comes to the success of the whole application. With Spring we usually made use of the RestTemplate to perform HTTP requests, but can now also use the Spring WebClient.
With the release of Spring Framework 5.0, the documentation of the RestTemplate says the following:
NOTE: As of 5.0, the non-blocking, reactive
org.springframework.web.reactive.client.WebClientoffers a modern alternative to theRestTemplatewith efficient support for both sync and async, as well as streaming scenarios. TheRestTemplatewill be deprecated in a future version and will not have major new features added going forward. See the WebClient section of the Spring Framework reference documentation for more details and example code.
To be future-ready, your Spring-based application should migrate to the reactive and non-blocking Spring WebClient for both its async & sync HTTP communication.
With this blog post, you'll learn how to effectively use the WebClient class for HTTP requests. Furthermore, we'll take a look at the WebClient configuration, filtering requests, and testing. For the demo application, we'll use Java 11 and Spring Boot 2.4.
Create and Configure the WebClient
To take advantage of the WebClient, we need the following dependency for our Spring Boot project:
| 1 2 3 4 | <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-webflux</artifactId> </dependency> |
The WebClient internally delegates to an HTTP client library (by default Reactor Netty), but others can be plugged in through a ClientHttpConnector.
When it comes to configuring resilient HTTP clients, connection/read/write timeouts are important to avoid long-running tasks. In addition, HTTP headers and cookies are essential for e.g. authentication or content negotiation. We can configure such timeouts when creating our WebClient instance.
When using the WebClient within a Spring Boot project, we can inject the auto-configured WebClient.Builder and create the instance using this builder. This auto-configured builder customizes the WebClient to, among other things, emit metrics about the HTTP response code and response time when the Spring Boot Actuator is on the classpath:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | @Bean public WebClient webClientFromBuilder(WebClient.Builder webClientBuilder) { HttpClient httpClient = HttpClient.create() .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 2_000) // millis .doOnConnected(connection -> connection .addHandlerLast(new ReadTimeoutHandler(2)) // seconds .addHandlerLast(new WriteTimeoutHandler(2))); //seconds return webClientBuilder .baseUrl(BASE_URL) .clientConnector(new ReactorClientHttpConnector(httpClient)) .defaultCookie("cookieKey", "cookieValue", "teapot", "amsterdam") .defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE) .defaultHeader(HttpHeaders.ACCEPT, MediaType.APPLICATION_JSON_VALUE) .defaultHeader(HttpHeaders.USER_AGENT, "I'm a teapot") .filter(ExchangeFilterFunctions.basicAuthentication("rieckpil", UUID.randomUUID().toString())) .filter(logRequest()) .filter(logResponse()) .build(); } |
The configuration above defines convenient defaults for the connect, read and write timeouts. Furthermore, we define default cookies, headers, and filters that are presented (unless we override them) for each request made with this WebClient bean. More about the filtering mechanism in one of the upcoming sections.
We can also create our own WebClient from scratch without using the pre-configured WebClient.Builder from Spring Boot:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | @Component public class SimpleWebClientConfiguration { private static final String BASE_URL = "https://jsonplaceholder.typicode.com"; private static final Logger logger = LoggerFactory.getLogger(SimpleApiClient.class); @Bean public WebClient webClientFromScratch() { return WebClient.builder() .baseUrl(BASE_URL) // similiar configuration .build(); } } |
HTTP GET Request Example With Spring WebClient
Once our WebClient is configured for a specific baseUrl, we can start performing HTTP requests.
As the internal WebClient architecture is designed for reactive and non-blocking applications, we either have to call .block() or rewrite our codebase to accept Mono<T> and Flux<T> as method return types.
A simple sync HTTP GET request with our previously configured WebClient looks like the following:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | @Service public class SimpleApiClient { private final WebClient defaultWebClient; // inject the configured WebClient @Bean from the configuration above public SimpleApiClient(WebClient defaultWebClient) { this.defaultWebClient = defaultWebClient; } public JsonNode getTodoFromAPI() { return this.defaultWebClient .get() .uri("/todos/1") .retrieve() .bodyToMono(JsonNode.class) .block(); } } |
To trigger the actual HTTP request, we can either use .retrieve() or .exchange() whereby the first method is the preferred way. With .exchange(), we have more control about the response but also more responsibilities. Take a look at this blog post or the relevant part of the documentation to understand their difference in detail.
All upcoming examples use the preferred .retrieve() method.
Handle Different HTTP Response Codes
By default, HTTP responses with 4xx (e.g. 404 for not found) or 5xx (e.g. 500 service unavailable) status codes result in an WebClientResponseException.
For modifying the way how errors are handled, we can use the onStatus method of Spring's WebClient to customize the resulting exception:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | @Service public class SimpleApiClient { private final WebClient defaultWebClient; public SimpleApiClient(WebClient defaultWebClient) { this.defaultWebClient = defaultWebClient; } public JsonNode getTodoFromAPI() { return this.defaultWebClient .get() .uri("/todos/1") .retrieve() .onStatus(HttpStatus::is4xxClientError, response -> { System.out.println("4xx error"); return Mono.error(new RuntimeException("4xx")); }) .onStatus(HttpStatus::is5xxServerError, response -> { System.out.println("5xx error"); return Mono.error(new RuntimeException("5xx")); }) .bodyToMono(JsonNode.class) .block(); } } |
HTTP GET Request Example With Spring WebClient
Performing HTTP POST request with the WebClient is as simple as the following:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | @Service public class SimpleApiClient { private final WebClient defaultWebClient; public SimpleApiClient(WebClient defaultWebClient) { this.defaultWebClient = defaultWebClient; } public JsonNode postToTodoAPI() { return this.defaultWebClient .post() .uri("/todos") .body(BodyInserters.fromValue("{ \"title\": \"foo\", \"body\": \"bar\", \"userId\": \"1\"}")) .retrieve() .bodyToMono(JsonNode.class) .block(); } } |
To put the request body in place, we can use the BodyInserters utility class and add any kind of Object, FormData, Publisher, MultipartData, Resource , etc.
Retry Configuration On Failure
For a more resilient architecture, we can configure retries for our WebClient . We can use the public methods of Mono and Flux for this purpose.
Let's say we want to retry two times (max. three HTTP calls) and add a small delay between each retry:
| 1 2 3 4 5 6 7 8 9 | public JsonNode getRetryTodoFromAPI() { return this.defaultWebClient .get() .uri("/todos/1") .retrieve() .bodyToMono(JsonNode.class) .retryWhen(Retry.fixedDelay(2, Duration.ofMillis(300))) .block(); } |
Furthermore, we can combine the retry mechanism with a timeout from Mono or Flux to provide a fallback value:
| 1 2 3 4 5 6 7 8 9 10 11 | public JsonNode getRetryTodoFromAPI() { return this.defaultWebClient .get() .uri("/todos/1") .retrieve() .bodyToMono(JsonNode.class) .retryWhen(Retry.max(5)) .timeout(Duration.ofSeconds(2), Mono.just(objectMapper.createObjectNode().put("message", "fallback"))) .block(); } |
Configure Filters for the Spring WebClient
If we have business logic or a configuration setup that applies to every HTTP request and response made with the WebClient, we can configure and chain multiple filters.
Logging the request/response might be such a requirement or adding authentication to the request. This is configured during the WebClient setup:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 | @Component public class SimpleWebClientConfiguration { private static final String BASE_URL = "https://jsonplaceholder.typicode.com"; private static final Logger logger = LoggerFactory.getLogger(SimpleApiClient.class); @Bean public WebClient webClientFromScratch() { // HttpClient configuration return WebClient.builder() .baseUrl(BASE_URL) .clientConnector(new ReactorClientHttpConnector(httpClient)) .filter(ExchangeFilterFunctions.basicAuthentication("rieckpil", UUID.randomUUID().toString())) .filter(logRequest()) .filter(logResponse()) .build(); } private ExchangeFilterFunction logRequest() { return (clientRequest, next) -> { logger.info("Request: {} {}", clientRequest.method(), clientRequest.url()); logger.info("--- Http Headers: ---"); clientRequest.headers().forEach(this::logHeader); logger.info("--- Http Cookies: ---"); clientRequest.cookies().forEach(this::logHeader); return next.exchange(clientRequest); }; } private ExchangeFilterFunction logResponse() { return ExchangeFilterFunction.ofResponseProcessor(clientResponse -> { logger.info("Response: {}", clientResponse.statusCode()); clientResponse.headers().asHttpHeaders() .forEach((name, values) -> values.forEach(value -> logger.info("{}={}", name, value))); return Mono.just(clientResponse); }); } private void logHeader(String name, List<String> values) { values.forEach(value -> logger.info("{}={}", name, value)); } } |
We can also provide global customizations for our WebClient instances using the WebClientCustomizer interface.
Writing Tests For Classes Using the WebClient
When it comes to writing unit tests for API clients, mocking the entire WebClient interaction with Mockito is usually not a good fit. A better approach is to start a local HTTP server and mock the HTTP responses from the remote system:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | class SimpleApiClientTest { private MockWebServer mockWebServer; private SimpleApiClient cut; // Class Under Test @BeforeEach void setup() throws IOException { this.mockWebServer = new MockWebServer(); this.mockWebServer.start(); this.cut = new SimpleApiClient(WebClient .builder() .baseUrl(mockWebServer.url("/").toString()) .build()); } @Test void testGetUserById() throws InterruptedException { MockResponse mockResponse = new MockResponse() .addHeader("Content-Type", "application/json; charset=utf-8") .setBody("{\"id\": 1, \"name\":\"write good tests\"}"); mockWebServer.enqueue(mockResponse); JsonNode result = cut.getTodoFromAPI(); assertEquals(1, result.get("id").asInt()); assertEquals("write good tests", result.get("name").asText()); RecordedRequest request = mockWebServer.takeRequest(); assertEquals("/todos/1", request.getPath()); } } |
More information and examples for this test setup can be found here.
Integration Testing With the WebTestClient
For efficient integrations tests that test HTTP endpoints of our own application, we can use the WebTestClient. It's similar to the TestRestTemplate.
This client offers the same functionality as the normal WebClient but has additional methods for convention assertions and expectations for the HTTP result. As soon as we use @SpringBootTest with a not-mocked Servlet environment, Spring Boot auto-configures the WebTestClient for us.
Let's assume we want to write an integration test for our HTTP endpoint /api/customers/1 to ensure our clients can search for a particular customer:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) class CustomerControllerIT { @Autowired private WebTestClient webTestClient; @Test void shouldReturnCustomerOne() { this.webTestClient .get() .uri("/api/customers/1") // the base URL is already configured for us .accept(MediaType.APPLICATION_JSON) .exchange() .expectStatus().isOk() .expectHeader().contentType(MediaType.APPLICATION_JSON) .expectBody() .jsonPath("$.customerId").isNotEmpty() .jsonPath("$.name").isNotEmpty(); } } |
The detailed usage of the WebTestClient is covered in another blog post.
Accessing OAuth2 Protected Resources with Spring WebClient
With Spring Security we get a full integration for OAuth2. Have a look at the following blog posts where I demonstrate how to enable the OAuth2 integration for the Spring WebClient:
- Spring WebClient OAuth2 Integration for Spring Web (Servlet)
- Spring WebClient OAuth2 Integration for Spring WebFlux
You can find the full source code for all examples on GitHub and more detailed information about the WebClient as part of the Spring documentation.
PS: If you want to achieve the same with Jakarta EE, take a look at the JAX-RS Client or the MicroProfile RestClient.
Have fun using the Spring WebFlux WebClient,
Phil
