hashicorp / hashicorp/consul

Consul does not display errors for failed HTTPS `checks`

Open
#21,372 1 comment 0 reactions 0 assignees View on GitHub
type/bug
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
1d 18h
Merged PRs (30d)
39

Description

### Nomad version
```
Nomad v1.8.0
BuildDate 2024-05-28T17:38:17Z
Revision 28b82e4b2259fae5a62e2ed47395334bea5a24c4
```

### Operating system and Environment details
```
$ uname -a
Linux ip-172-31-2-172 6.5.0-1020-aws hashicorp/nomad#20~22.04.1-Ubuntu SMP Wed May 1 16:10:50 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
```

### Issue
If the service.check configured to check https without `tls_server_name`:
```tf
service {
tags = ["wi-nginx","https"]
name = "wi-nginx"
port = "https"

provider = "consul"
check {
name = "wi-nginx_https_health"
type = "http"
protocol = "https"
# tls_server_name = "${var.project_name}.example.com"
method = "GET"
path = "/_.gif"
interval = "10s"
timeout = "2s"

check_restart {
limit = 3
grace = "90s"
ignore_warnings = false
}
}
}
```
and the certificate on the Nginx site does not support SAN `wi-nginx.service.internal` then there is no way to see why the check fails.

### Reproduction steps
See the config above.

#### Expected Result
Nomad should log somewhere if the check fails.

#### Actual Result
I can not find any error messages regarding HTTPS check.

![image](https://github.com/hashicorp/nomad/assets/139780495/b8099a98-9b0f-4a7e-8cf0-9dad6334b19e)

Also this is strange to see the raw IP address when `tls_server_name` was configured:
![image](https://github.com/hashicorp/nomad/assets/139780495/73041330-b66d-49ee-853e-d6a67009c14d)

When this request actually fails:
```
$ wget https://172.31.2.172:443/_.gif
Connecting to 172.31.2.172:443 (172.31.2.172:443)
081B06BD517D0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1889:
ssl_client: SSL_connect
wget: error getting response: Connection reset by peer
```

#### Proposition
Nomad should distinguish the error returned from the check and the failed check.
In my case the check even did not run (server was not reached, though it function well).
From the inside `nginx` container I ran:
```
$ wget https://localhost/_.gif
Connecting to localhost ([::1]:443)
08BBE122FC7B0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1889:
ssl_client: SSL_connect
```
From here we can see that the request failed. Nginx does not have any errors in its `/var/log/nginx/error.log` file.

Contributor guide

Open the contributing guide

Research direction

Start with the Nomad service.check configuration shown in the issue and reproduce the HTTPS failure using the provided certificate and wget output. Trace where the check result is reported, then ensure the underlying HTTPS error is visible and distinguish it from the failed-check status.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.