spring-cloud / spring-cloud/spring-cloud-gateway

Unable to Stream Chunked Response

Open
#3,505 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

waiting-for-triage
Dominant language
Java
Stars
4.9k
Forks
3.5k
Avg merge
20h 57m
Merged PRs (30d)
8

Description

Hello,

I am using spring cloud gateway mvc: 4.1.5
When making requests that expects a streaming chunked response - where each chunk is returned to the api caller as soon as a chunk is ready.
The gateway returns the response in different batches, each with multiple chunks at the same time.

I found a similar issue raised here: https://github.com/spring-cloud/spring-cloud-gateway/issues/2868

I don't think its that streaming is not supported as curl is return the response from the gateway in batches - instead of the entire response being returned all at once.
It could be that the gateway is somehow caching the response chunks. Is there configuration that can be done on the gateway to resolve it?

Sample
I have tried a variety of different approaches. Including the examples in the linked issue.

My original approach using just handler functions:

defaultRoute = route("some-route")
                .route(path(this.targetApiPrefix), https(uri))
                .build();

Attempt to use of StreamingResponseBody with ProxyExchange

@PostMapping("/api/v1/chat")
public ResponseEntity<?> proxyChatApi(
        @RequestHeader(HttpHeaders.AUTHORIZATION) String authorizationHeader,
        @RequestBody Map body,
        ProxyExchange<?> proxyExchange) throws Exception {

    proxyExchange.header(HttpHeaders.AUTHORIZATION, authorizationHeader);
    proxyExchange.body(body);
    var response = proxyExchange.uri(this.uri).post();

    StreamingResponseBody stream = outputStream -> {
        StreamUtils.copy(response.getBody().getContentAsByteArray(), outputStream);
    };
    return new ResponseEntity(stream, response.getStatusCode());
}

Attempt to return a Flux<?> with ProxyExchange - I am not very familiar with the reactive way so pardon me if this seems incorrect:

@PostMapping("/api/v1/chat")
public Flux<?> proxyChatApi(
        @RequestHeader(HttpHeaders.AUTHORIZATION) String authorizationHeader,
        @RequestBody Map body,
        ProxyExchange<?> proxyExchange) throws Exception {
    var uri = new URI(this.uri);
    proxyExchange.header(HttpHeaders.AUTHORIZATION, authorizationHeader);
    proxyExchange.body(body);
    var response = proxyExchange.uri(uri.toString()).post();
    return Flux.from(response);
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the behavior with the Spring Cloud Gateway MVC 4.1.5 route and compare it with the linked issue. Review the ProxyExchange and StreamingResponseBody examples in the report, then determine whether the gateway or proxy path batches chunks. Done means documenting or implementing a verified way for each response chunk to reach the API caller promptly.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring, spring-boot
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.