[Heartbeat] Silent remote error: tls: handshake failure
- Dominant language
- Go
- Stars
- 12.7k
- Forks
- 5k
- Avg merge
- 2d 15m
- Merged PRs (30d)
- 385
Description
After upgrading Filebeat to 8.19.2, HTTP monitors configured with `ssl.verification_mode: none` incorrectly report as down, and the API `/internal/uptime/monitor/status?monitorId=&dateStart=now-15m&dateEnd=now` returns the following error:
```json
"error": {
"message": "Get \"https://example.com:80/healthcheck\": remote error: tls: handshake failure",
"type": "io"
},
"url": {
"path": "/healthcheck",
"full": "https://example.com:80/healthcheck",
"scheme": "https",
"domain": "example.com",
"port": 80
}
```
There is nothing relevant in the logs, even at debug level. We could track it down to these changes:
- https://github.com/golang/go/issues/66512#issuecomment-2019511690
>In Go 1.22 RSA key exchange based cipher suites were removed from the default list, but can be re-added with the GODEBUG setting tlsrsakex=1.
- https://github.com/elastic/beats/pull/40082#issuecomment-2227716350
>By default, cipher suites without ECDHE support are no longer offered by either clients or servers during pre-TLS 1.3 handshakes. This change can be reverted with the tlsrsakex=1 GODEBUG setting.
While setting the `GODEBUG=tlsrsakex=1` environment variable resolves the issue, the error is really not obvious as Heartbeat doesn't log anything.
Possible ways to address this bug include:
- Logging the TLS handshake issue at the error level and providing guidance on how to resolve it.
- Documenting this breaking change in the official documentation.
- Alternatively, setting `GODEBUG=tlsrsakex=1` by defautl
Contributor guide
Research direction
Start at Heartbeat's HTTP monitor handling for the reported TLS handshake failure and reproduce the example with ssl.verification_mode: none against an endpoint using the affected cipher negotiation. Compare the behavior with GODEBUG=tlsrsakex=1; done should be an agreed fix that makes the failure visible and actionable, documents the compatibility change, or explicitly chooses another listed remedy.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- networking, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100