OpenAPITools / OpenAPITools/openapi-generator

[REQ][Java] Access to WebClient.RequestBodySpec

Open
#20,966 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Enhancement: Feature
Dominant language
Java
Stars
26.8k
Forks
7.7k
PR metrics pending

Description

Is your feature request related to a problem? Please describe.

In my application the response body for a request is a very large JSON which leads to following DataBufferLimitException when it tries to parse the body to a Java object:

Stacktrace 
Caused by: org.springframework.core.io.buffer.DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144
	at org.springframework.core.io.buffer.LimitedDataBufferList.raiseLimitException(LimitedDataBufferList.java:99)
	Suppressed: The stacktrace has been enhanced by Reactor, refer to additional information below: 
Assembly trace from producer [reactor.core.publisher.MonoCollect] :
	reactor.core.publisher.Flux.collect(Flux.java:3722)
	org.springframework.core.io.buffer.DataBufferUtils.join(DataBufferUtils.java:677)
Error has been observed at the following site(s):
	*__________Flux.collect ⇢ at org.springframework.core.io.buffer.DataBufferUtils.join(DataBufferUtils.java:677)
	|_          Mono.filter ⇢ at org.springframework.core.io.buffer.DataBufferUtils.join(DataBufferUtils.java:678)
	|_             Mono.map ⇢ at org.springframework.core.io.buffer.DataBufferUtils.join(DataBufferUtils.java:679)
	|_     Mono.doOnDiscard ⇢ at org.springframework.core.io.buffer.DataBufferUtils.join(DataBufferUtils.java:680)
	|_         Mono.flatMap ⇢ at org.springframework.http.codec.json.AbstractJackson2Decoder.lambda$decodeToMono$3(AbstractJackson2Decoder.java:197)
	*__Mono.deferContextual ⇢ at org.springframework.http.codec.json.AbstractJackson2Decoder.decodeToMono(AbstractJackson2Decoder.java:192)
	|_           checkpoint ⇢ Body from POST https://dmsresteai-test-dmsresteai.apps.capk.muenchen.de/api/objectAndImportToInbox [DefaultClientResponse]
Original Stack Trace:
		at org.springframework.core.io.buffer.LimitedDataBufferList.raiseLimitException(LimitedDataBufferList.java:99)
		at org.springframework.core.io.buffer.LimitedDataBufferList.updateCount(LimitedDataBufferList.java:92)
		at org.springframework.core.io.buffer.LimitedDataBufferList.add(LimitedDataBufferList.java:58)
		at reactor.core.publisher.MonoCollect$CollectSubscriber.onNext(MonoCollect.java:103)
		at reactor.core.publisher.FluxPublish$PublishSubscriber.drain(FluxPublish.java:571)
		at reactor.core.publisher.FluxPublish$PublishSubscriber.onNext(FluxPublish.java:310)
		at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107)
		at reactor.core.publisher.FluxMapFuseable$MapFuseableConditionalSubscriber.onNext(FluxMapFuseable.java:299)
		at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
		at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
		at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
		at reactor.adapter.JdkFlowAdapter$SubscriberToRS.onNext(JdkFlowAdapter.java:150)
		at java.net.http/jdk.internal.net.http.ResponseSubscribers$PublishingBodySubscriber.onNext(ResponseSubscribers.java:1028)
		at java.net.http/jdk.internal.net.http.ResponseSubscribers$PublishingBodySubscriber.onNext(ResponseSubscribers.java:868)
		at java.net.http/jdk.internal.net.http.common.HttpBodySubscriberWrapper.onNext(HttpBodySubscriberWrapper.java:391)
		at java.net.http/jdk.internal.net.http.common.HttpBodySubscriberWrapper.onNext(HttpBodySubscriberWrapper.java:49)
		at java.net.http/jdk.internal.net.http.ResponseContent$ChunkedBodyParser.accept(ResponseContent.java:237)
		at java.net.http/jdk.internal.net.http.ResponseContent$ChunkedBodyParser.accept(ResponseContent.java:131)
		at java.net.http/jdk.internal.net.http.Http1Response$BodyReader.handle(Http1Response.java:707)
		at java.net.http/jdk.internal.net.http.Http1Response$BodyReader.handle(Http1Response.java:635)
		at java.net.http/jdk.internal.net.http.Http1Response$Receiver.accept(Http1Response.java:527)
		at java.net.http/jdk.internal.net.http.Http1Response$BodyReader.tryAsyncReceive(Http1Response.java:665)
		at java.net.http/jdk.internal.net.http.Http1AsyncReceiver.flush(Http1AsyncReceiver.java:233)
		at java.net.http/jdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask.run(SequentialScheduler.java:182)
		at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:149)
		at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:207)
		at java.net.http/jdk.internal.net.http.HttpClientImpl$DelegatingExecutor.execute(HttpClientImpl.java:177)
		at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:282)
		at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:251)
		at java.net.http/jdk.internal.net.http.Http1AsyncReceiver.asyncReceive(Http1AsyncReceiver.java:468)
		at java.net.http/jdk.internal.net.http.Http1AsyncReceiver$Http1TubeSubscriber.onNext(Http1AsyncReceiver.java:589)
		at java.net.http/jdk.internal.net.http.Http1AsyncReceiver$Http1TubeSubscriber.onNext(Http1AsyncReceiver.java:546)
		at java.net.http/jdk.internal.net.http.common.SSLTube$DelegateWrapper.onNext(SSLTube.java:210)
		at java.net.http/jdk.internal.net.http.common.SSLTube$SSLSubscriberWrapper.onNext(SSLTube.java:492)
		at java.net.http/jdk.internal.net.http.common.SSLTube$SSLSubscriberWrapper.onNext(SSLTube.java:295)
		at java.net.http/jdk.internal.net.http.common.SubscriberWrapper$DownstreamPusher.run1(SubscriberWrapper.java:316)
		at java.net.http/jdk.internal.net.http.common.SubscriberWrapper$DownstreamPusher.run(SubscriberWrapper.java:259)
		at java.net.http/jdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask.run(SequentialScheduler.java:182)
		at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:149)
		at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:207)
		at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:280)
		at java.net.http/jdk.internal.net.http.common.SequentialScheduler.runOrSchedule(SequentialScheduler.java:233)
		at java.net.http/jdk.internal.net.http.common.SubscriberWrapper.outgoing(SubscriberWrapper.java:232)
		at java.net.http/jdk.internal.net.http.common.SubscriberWrapper.outgoing(SubscriberWrapper.java:198)
		at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate$Reader.processData(SSLFlowDelegate.java:465)
		at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate$Reader$ReaderDownstreamPusher.run(SSLFlowDelegate.java:283)
		at java.net.http/jdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask.run(SequentialScheduler.java:182)
		at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:149)
		at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:207)
		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
		at java.base/java.lang.Thread.run(Thread.java:1583)

This is a known behavior within Spring WebClient: https://github.com/spring-projects/spring-framework/issues/25838

Through WebClient.RequestBodySpec it would be possible to get the body as InputStream (see https://www.baeldung.com/spring-reactive-read-flux-into-inputstream#bodyextractors-databufferutils) which I then could correctly parse myself.

Describe the solution you'd like

It would be great if there was a way to get a WebClient.RequestBodySpec that could be used to process the response body in a more customizable way. (In my case as InputStream)

Describe alternatives you've considered

Additional context

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the WebClient.RequestBodySpec API and the referenced Spring Framework issue 25838 to understand the response-body limitation and possible customization point. Then trace how this repository generates Java WebClient clients; done would mean generated clients provide a documented way to process large response bodies without the default buffering limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.