spring-cloud / spring-cloud/spring-cloud-vault

VaultReactive (health) does not recover after some vault server issues.

Open
#654 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-feedback
Dominant language
Java
Stars
291
Forks
152
Avg merge
11h 24m
Merged PRs (30d)
3

Description

We are using servicediscovery (Netflix/Eureka) and have several services that are using spring-cloud-vault. After some issues with our vault server/network the health check of the spring-cloud-vault does not recover by him self. The servicediscovery admin page noted that vaultReactive is down.

The following error was on the admin panel:
org.springframework.vault.authentication.VaultLoginException: Cannot retrieve VaultToken from authentication chain; nested exception is org.springframework.web.reactive.function.client.WebClientResponseException$ServiceUnavailable: 503 Service Unavailable from POST https://....

Only a restart of the services solved the problem. I think it would be an improvement when the health check of spring-cloud-vault would recover by him self so that no restart of all services is required.

There is an other exception when the service is shutted down:
Invocation of destroy method failed on bean with name 'reactiveVaultSessionManager': org.springframework.vault.authentication.VaultLoginException: Cannot retrieve VaultToken from authentication chain; nested exception is org.springframework.web.reactive.function.client.WebClientResponseException$ServiceUnavailable: 503 Service Unavailable from POST https://....

It seems to me that the connection is somehow broken. The vault server is available because after restart it works fine.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing a Vault or network failure that returns 503 during authentication, then inspect the reactiveVaultSessionManager lifecycle and the VaultReactive health check behavior. Verify that the health check recovers when Vault becomes available again without restarting the service, and that shutdown no longer reports the VaultLoginException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
backend, cloud, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.