RetryGatewayFilter causing out of direct memory errors

Describe the bug
When using the gateway in combination with a backend application that accepts large multipart file uploads (tested with 250MB) we encountered the situation that the direct memory overflowed. The filter is configured to only retry GET requests but the caching mechanism intercepts the complete route and therefore also caches the multipart in the buffers. After completion the buffers are not released.

For the moment we worked around by using a dedicated GET route before all other HTTP methods.

{"@timestamp":"2020-11-06T14:46:49.835Z","@version":"1","message":"[7de5326e-4919504]  500 Server Error for HTTP POST \"/web/uploads/temp\"","logger_name":"org.springframework.boot.autoconfigure.web.reactive.error.AbstractErrorWebExceptionHandler","thread_name":"reactor-http-epoll-1","level":"ERROR","level_value":40000,"stack_trace":"reactor.netty.ReactorNetty$InternalNettyException: java.lang.OutOfMemoryError: Cannot reserve 16777216 bytes of direct buffer memory (allocated: 1057308680, limit: 1073741824)
 \tSuppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
 Error has been observed at the following site(s):
 \t|_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
 \t|_ checkpoint ⇢ org.springframework.security.web.server.authorization.AuthorizationWebFilter [DefaultWebFilterChain]
 \t|_ checkpoint ⇢ org.springframework.security.web.server.authorization.ExceptionTranslationWebFilter [DefaultWebFilterChain]
 \t|_ checkpoint ⇢ org.springframework.security.web.server.savedrequest.ServerRequestCacheWebFilter [DefaultWebFilterChain]
 \t|_ checkpoint ⇢ org.springframework.security.web.server.context.SecurityContextServerWebExchangeWebFilter [DefaultWebFilterChain]
 \t|_ checkpoint ⇢ org.springframework.security.config.web.server.ServerHttpSecurity$OAuth2ResourceServerSpec$BearerTokenAuthenticationWebFilter [DefaultWebFilterChain]
 \t|_ checkpoint ⇢ org.springframework.security.web.server.authentication.AuthenticationWebFilter [DefaultWebFilterChain]
 \t|_ checkpoint ⇢ org.springframework.security.web.server.context.ReactorContextWebFilter [DefaultWebFilterChain]
 \t|_ checkpoint ⇢ org.springframework.security.web.server.header.HttpHeaderWriterWebFilter [DefaultWebFilterChain]
 \t|_ checkpoint ⇢ org.springframework.security.config.web.server.ServerHttpSecurity$ServerWebExchangeReactorContextWebFilter [DefaultWebFilterChain]
 \t|_ checkpoint ⇢ org.springframework.security.web.server.WebFilterChainProxy [DefaultWebFilterChain]
 \t|_ checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
 \t|_ checkpoint ⇢ HTTP POST \"/web/uploads/temp\" [ExceptionHandlingWebHandler]
 Stack trace:
 Caused by: java.lang.OutOfMemoryError: Cannot reserve 16777216 bytes of direct buffer memory (allocated: 1057308680, limit: 1073741824)
 \tat java.base/java.nio.Bits.reserveMemory(Unknown Source)
 \tat java.base/java.nio.DirectByteBuffer.<init>(Unknown Source)
 \tat java.base/java.nio.ByteBuffer.allocateDirect(Unknown Source)
 \tat io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:645)
 \tat io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:621)
 \tat io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:204)
 \tat io.netty.buffer.PoolArena.tcacheAllocateNormal(PoolArena.java:188)
 \tat io.netty.buffer.PoolArena.allocate(PoolArena.java:138)
 \tat io.netty.buffer.PoolArena.allocate(PoolArena.java:128)
 \tat io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:378)
 \tat io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
 \tat io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
 \tat io.netty.channel.unix.PreferredDirectByteBufAllocator.ioBuffer(PreferredDirectByteBufAllocator.java:53)
 \tat io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:114)
 \tat io.netty.channel.epoll.EpollRecvByteAllocatorHandle.allocate(EpollRecvByteAllocatorHandle.java:75)
 \tat io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:780)
 \tat io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475)
 \tat io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
 \tat io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
 \tat io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
 \tat io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
 \tat java.base/java.lang.Thread.run(Unknown Source)
 "}

Sample
Reproducible with a plain spring cloud gateway project (spring boot 2.3.5 / Hoxton.SR8) with a connected spring application that accepts multipart file uploads. The associated application config is the following.

server:
  port:
    8001

spring:
  application:
    name: gateway
  cloud:
    gateway:
      routes:
        - id: test
          uri: lb://test
          predicates:
            - Path=/web/**,/api/**,/webdav/**
          filters:
            - PreserveHostHeader
            - name: Retry
              args:
                retries: 3
                statuses: BAD_GATEWAY
                exceptions:
                  - java.net.ConnectException
          order: 1
      loadbalancer:
        use404: true
      x-forwarded:
        enabled: false
        proto-enabled: false

    loadbalancer:
      ribbon:
        enabled: false
eureka:
  client:
    registry-fetch-interval-seconds: 5

Start by reproducing the multipart upload with the shown Spring Cloud Gateway configuration and inspect the Retry filter's handling of non-GET requests and request buffers. Done means the 250MB upload no longer causes direct-memory exhaustion when retries are configured only for GET requests; no source file or test is named in the issue.