spring-cloud / spring-cloud/spring-cloud-config

Health check from other service configured with retry policy to config server can hang.

Open
#510 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

closable
Dominant language
Java
Stars
2k
Forks
1.3k
Avg merge
2d 59m
Merged PRs (30d)
16

Description

We encountered some strange behavior when we have a service, which is configured to contact the config-service with a retry policy (for instance try for 30 minutes) with regards to that service health check page.

Looks like the same retry policy is used for updating the health information of the config service in that service. However, when the config service is down, this check is retrying and retrying. At that time, the /health page of the service hangs and doesn't show.

We use external automated monitoring on /health to identify the health status. In the above case, it marks the service as down, while it isn't.

I'd guess the health check towards the config server should not take into account the configured retry policy and just show DOWN instantly when it is not there.

Kind regards,
Wouter

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the interaction between the service /health endpoint, the config-server health check, and a long configured retry policy while the config server is unavailable. Trace the health-check and retry-policy handling in the Spring Cloud Config client; it is done when /health responds promptly and reports the config service as down instead of hanging.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring, spring-boot
Domain
backend, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.