spring-cloud / spring-cloud/spring-cloud-commons

Removal of unhealthy server pods on java.net.SocketTimeoutException: Connect Timeout

Open
#1,059 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

waiting-for-triage
Dominant language
Java
Stars
751
Forks
744
Avg merge
1d 14h
Merged PRs (30d)
9

Description

Hello Team,

Is your feature request related to a problem? Please describe.
We are using Spring Cloud Load Balancer version 2.0.5 on Kubernetes v1.20.0.
When a server pod terminates the requests that are load balanced towards the pod start getting the java.net.SocketTimeoutException: Connect Timeout exception.

Describe the solution you'd like
On a event of a Connect timeout the pod should be removed from the service list immediately.
This will still result in 1st packet getting the exception but subsequent requests will load balanced among remaining healthy pods.

Describe alternatives you've considered
To solve this problem we had enabled health-check.refresh-instances-interval but until the health check timeout the requests are routed to the deleted pod.

Additional context
Please let us know if there is a way to remove the service instances based on Connect timeout exception or is there a feature which is planned to achieve this.

Thanks and Regards,
Vijit

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing Spring Cloud Load Balancer's instance lifecycle and how java.net.SocketTimeoutException connection failures are handled. Define the required behavior for removing an unhealthy Kubernetes pod from the service list, then verify that subsequent requests avoid it while the first failed request remains expected.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kubernetes, spring
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.