Expand Authors Top

If you have a few years of experience in the Java ecosystem and you’d like to share that with the community, have a look at our Contribution Guidelines.

Expanded Audience – Frontegg – Security (partner)
announcement - icon User management is very complex, when implemented properly. No surprise here.

Not having to roll all of that out manually, but instead integrating a mature, fully-fledged solution - yeah, that makes a lot of sense.
That's basically what Frontegg is - User Management for your application. It's focused on making your app scalable, secure and enjoyable for your users.
From signup to authentication, it supports simple scenarios all the way to complex and custom application logic.

Have a look:

>> Elegant User Management, Tailor-made for B2B SaaS

November Discount Launch 2022 – Top
We’re finally running a Black Friday launch. All Courses are 30% off until tomorrow:

>> GET ACCESS NOW

NPI – Lightrun – Spring (partner)

We rely on other people’s code in our own work. Every day. It might be the language you’re writing in, the framework you’re building on, or some esoteric piece of software that does one thing so well you never found the need to implement it yourself.

The problem is, of course, when things fall apart in production - debugging the implementation of a 3rd party library you have no intimate knowledge of is, to say the least, tricky. It’s difficult to understand what talks to what and, specifically, which part of the underlying library is at fault.

Lightrun is a new kind of debugger.

It's one geared specifically towards real-life production environments. Using Lightrun, you can drill down into running applications, including 3rd party dependencies, with real-time logs, snapshots, and metrics. No hotfixes, redeployments, or restarts required.

Learn more in this quick, 5-minute Lightrun tutorial:

>> The Essential List of Spring Boot Annotations and Their Use Cases

1. Introduction

In this tutorial, we'll explore why we may see DataBufferLimitException in a Spring Webflux application. We'll then take a look at the various ways we can resolve the same.

2. Understanding the Problem

Let's understand the problem first before jumping to the solution.

2.1. What's DataBufferLimitException?

Spring WebFlux limits buffering of data in-memory in codec to avoid application memory issues. By default, this is configured to 262,144 bytes. When this isn't enough for our use case, we’ll end up with the DataBufferLimitException.

2.2. What's a Codec?

The spring-web and spring-core modules provide support for serializing and deserializing byte content to and from higher-level objects through non-blocking I/O with reactive stream back pressure. Codecs offer an alternative to Java serialization. One advantage is that, typically, objects need not implement Serializable.  

3. Server Side

Let's first look at how DataBufferLimitException plays out from a server perspective.

3.1. Reproducing the Issue

Let's try to send a JSON payload of size 390 KB to our Spring Webflux server application to create the exception. We'll use the curl command to send a POST request to our server:

curl --location --request POST 'http://localhost:8080/1.0/process' \
  --header 'Content-Type: application/json' \
  --data-binary '@/tmp/390KB.json'

As we can see, the DataBufferLimitException is thrown:

org.springframework.core.io.buffer.DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144
  at org.springframework.core.io.buffer.LimitedDataBufferList.raiseLimitException(LimitedDataBufferList.java:99) ~[spring-core-5.3.23.jar:5.3.23]
  Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
  *__checkpoint ⇢ HTTP POST "/1.0/process" [ExceptionHandlingWebHandler]

3.2. Solution via Properties

The easiest solution would be to configure the application property spring.codec.max-in-memory-size. Let's add the following to our application.yaml file:

spring:
    codec:
        max-in-memory-size: 500KB

With that, we should now be able to buffer payloads larger than 500 KB in our application.

3.3. Solution via Code

Alternatively, we can use the WebFluxConfigurer interface to configure the same thresholds. To do this, we'll add a new configuration class, WebFluxConfiguration:

@Configuration
public class WebFluxConfiguration implements WebFluxConfigurer {
    @Override
    public void configureHttpMessageCodecs(ServerCodecConfigurer configurer) {
        configurer.defaultCodecs().maxInMemorySize(500 * 1024);
    }
}

This approach will also provide us with the same results.

4. Client Side

Let's now switch gears to look at the client-side behavior.

4.1. Reproducing the Issue

We'll try to reproduce the same behavior with Webflux's WebClient. Let's create a handler that calls the server with a payload of 390 KB:

public Mono<Users> fetch() {
    return webClient
      .post()
      .uri("/1.0/process")
      .body(BodyInserters.fromPublisher(readRequestBody(), Users.class))
      .exchangeToMono(clientResponse -> clientResponse.bodyToMono(Users.class));
}

We see again that the same exception is thrown but this time due to the webClient trying to send a larger payload than allowed:

org.springframework.core.io.buffer.DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144
  at org.springframework.core.io.buffer.LimitedDataBufferList.raiseLimitException(LimitedDataBufferList.java:99) ~[spring-core-5.3.23.jar:5.3.23]
  Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
    *__checkpoint ⇢ Body from POST http://localhost:8080/1.0/process [DefaultClientResponse]
    *__checkpoint ⇢ Handler com.baeldung.[email protected]428eedd9 [DispatcherHandler]
    *__checkpoint ⇢ HTTP POST "/1.0/trigger" [ExceptionHandlingWebHandler]

4.2. Solution via Properties

Again, the easiest solution is to configure the application property spring.codec.max-in-memory-size. Let's add the following to our application.yaml file:

spring:
    codec:
        max-in-memory-size: 500KB

With that, we should now be able to send payloads larger than 500 KB from our application. It's worth noting that this configuration gets applied to the entire application, which means to all web clients and the server itself.

Hence, if we want to configure this limit only for specific web clients, then this won't be an ideal solution. Additionally, there is a caveat with this approach. The builder used to create the WebClients must be auto-wired by Spring like the below:

@Bean("webClient")
public WebClient getSelfWebClient(WebClient.Builder builder) {
    return builder.baseUrl(host).build();
}

4.3. Solution via Code

We've also got a programmatic way to configure the web clients to achieve this goal. Let's create a WebClient with the following configuration:

@Bean("progWebClient")
    public WebClient getProgSelfWebClient() {
        return WebClient
          .builder()
          .baseUrl(host)
          .exchangeStrategies(ExchangeStrategies
	  .builder()
	  .codecs(codecs -> codecs
            .defaultCodecs()
            .maxInMemorySize(500 * 1024))
	    .build())
          .build();
}

And with that, we should now be able to successfully send payloads higher than 500 KB using our web client.

5. Conclusion

In this article, we understood what DataBufferLimitException is and looked at how to fix them on both the server and client sides. We looked at two approaches for both, firstly based on properties configuration and secondly programmatically. We hope this exception won't be a trouble for you anymore.

As always, the complete code is available over on GitHub.

November Discount Launch 2022 – Bottom
We’re finally running a Black Friday launch. All Courses are 30% off until tomorrow:

>> GET ACCESS NOW

Generic footer banner
Comments are closed on this article!