SCG (MVC) – Retry Filter & Load Balancer Retry Not Working For LoadBalancer Failures

Describe the bug

I'm trying to configure retry logic in SCG to retry failed requests on the next available instance provided by Spring Cloud LoadBalancer.

I’ve added a Retry filter in my route configuration, and it is not retrying when the downstream service is down but available in eureka.

I also included the retry configuration for Spring Cloud LoadBalancer from the documentation, but it appears retry on failure isn't supported in SCG MVC, even though the documentation suggests otherwise.

• https://docs.spring.io/spring-cloud-commons/reference/spring-cloud-commons/common-abstractions.html#retrying-failed-requests

application.yaml

        routes:
          - id: demoservice
            uri: lb://demoservice
            predicates:
              - Path=/demoservice/**
            filters:
              - StripPrefix=1
              - name: Retry
                args:
                  retries: 3
                  series: SERVER_ERROR
                  methods: GET,POST
                  cacheBody: true
                  exceptions:
                    - java.net.ConnectException
                    - java.util.concurrent.TimeoutException

loadbalancer:
  retry:
    enabled: true
    maxRetriesOnSameServiceInstance: 0
    maxRetriesOnNextServiceInstance: 3
    retryableStatusCodes: 503,500,501,502,504
    avoidPreviousInstance: true`

spring cloud 2024.0.2
spring boot 3.4.8

Steps to Reproduce

1. Start Eureka Server
2. Start two instances of demoservice (registered with Eureka)
3. Start the Spring Cloud Gateway MVC application

Verify the setup:

    Run: curl http://localhost:8080/demoservice/hello

    Response should return successfully (load balanced between both instances)

Simulate instance failure:

    Stop one of the demoservice instances (do not deregister from Eureka immediately)

Test again with curl:

    Run: curl http://localhost:8080/demoservice/hello multiple times

    You will observe intermittent failures (50%)

    the gateway is routing to the down instance and not retrying the request on the next available instance

Attached is a minimal project to reproduce the issue:
lb-retry-project.zip

Start by running the attached lb-retry-project.zip with Eureka, two demoservice instances, and the Spring Cloud Gateway MVC application. Exercise the lb://demoservice route with curl after stopping one instance, then trace the MVC Retry filter and Spring Cloud LoadBalancer retry path. Done means failed requests retry on another available instance instead of intermittently returning errors.