spring-projects / spring-projects/spring-framework

AbstractJackson2Decoder.decodeToMono should use non-blocking parser

Open
#25,838 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

in: web status: pending-design-work type: enhancement
Dominant language
Java
Stars
60.2k
Forks
38.8k
Avg merge
5d 2h
Merged PRs (30d)
27

Description

AbstractJackson2Decoder.decodeToMono currently uses DataBufferUtils.join to join all DataBuffers, converts the result to an input stream, and then uses jackson's blocking parser. This approach reads all DataBuffers for the entire payload into memory at the same time. For large payloads, this is not memory efficient.

On the other hand, AbstractJackson2Decoder.decode uses jackson's non-blocking parser. This approach does not require reading all buffers into memory at the same time. i.e. each buffer can be processed individually, and released as it is parsed.

For larger payloads, I think it would be better if decodeToMono also used jackson's non-blocking parser. This would allow each data buffer to be processed individually and released, rather than having to read the entire payload into memory.

Using the non-blocking parser would allow the maxInMemorySize to be increased to accommodate occasional large payloads, and allow the server to be more memory efficient when reading them.

It would also bring the behavior of JSON parsing in WebFlux more inline with the behavior of JSON parsing in WebMVC. WebMVC's AbstractJackson2HttpMessageConverter does not read the entire payload into memory and does not enforce a maxInMemorySize.

FWIW, I started looking into this when I hit a DataBufferLimitException for some occasional large payloads. I'd like to increase the maxInMemorySize, but then I noticed the consequence of loading everything into memory at the same time, which was a bit surprising since WebMVC doesn't have this limit.

What are your thoughts?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in spring-web/src/main/java/org/springframework/http/codec/json/AbstractJackson2Decoder.java, comparing decodeToMono, which uses DataBufferUtils.join and an input stream, with decode, which uses Jackson's non-blocking parser. The change is complete when decodeToMono processes and releases buffers individually without loading the entire payload into memory, while retaining the intended maxInMemorySize behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.