Certificate errors do not affect node health
- Dominant language
- Go
- Stars
- 30.1k
- Forks
- 4.6k
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 39
Description
#### Overview of the Issue
I have a cluster of consul servers that are using TLS encryption. All nodes were joined and communicating fine. When I let the certificates expire, the logs are full of errors but the nodes, entries for the "consul" service and the ui all report healthy status. I expected them to show unhealthy.
#### Reproduction Steps
Steps to reproduce this issue, eg:
1. Create a cluster with 3 server nodes, gossip encryption and enable tls between them with short-dated certificates.
2. Confirm the cluster is working properly and the nodes are all communicating.
3. Wait until the certificates expire.
4. The ui reports everything is healthy but the logs contain rows of
[ERROR] agent.server.rpc: failed to read byte: conn=from=x.x.x.x:yyyy error="remote error: tls: bad certificate"
### Consul info for both Client and Server
Consul 1.9.4
### Operating system and Environment details
ubuntu 18.04 on aws
Contributor guide
Research direction
Start by reproducing the issue with a three-server cluster using short-lived TLS certificates, then compare the certificate errors in the logs with node, consul service, and UI health reports. Trace how those health statuses are derived and make the expected unhealthy state verifiable after certificate expiry.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems, observability, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100