clustering: peer TLS certificates are validated against their IP address instead of their hostname

What did you do?

set demon option --cluster.peer $hostname:9094, where $hostname is the fqdn of the machine itself (it is the first node in the cluster)

What did you expect to see?

TLS certificate validation should check that the cert validates against the hostname given to the peer option.

What did you see instead? Under which circumstances?

The service fails to start and issues an error about the certificate not validating against the peer's IP address.

It is not common practice to include an SAN entry for an IP address. Certificates are meant to be bound to the service on a hostname, not an IP address. So being told that we need an IP SAN is just weird.

Please note that this issue was already reported in #3159 but it was closed because of inactivity, without really addressing the problem. In that issue @stuwilkins pointed out a workaround (thanks so much for that) but my take is that it should not be necessary to configure the "tls_client" part of the config to force the hostname and alertmanager should instead validate against the hostname that was given to it as an option.

System information

Debian trixie

Alertmanager version

alertmanager, version 0.28.1+ds (branch: debian/sid, revision: 0.28.1+ds-1)
  build user:       team+pkg-go@tracker.debian.org
  build date:       20250309-13:50:43
  go version:       go1.24.1
  platform:         linux/amd64
  tags:             unknown

installed via debian archives

Alertmanager configuration file

Prometheus version

Prometheus configuration file

Logs

~~~
Mar 25 19:42:05 prometheus-04 prometheus-alertmanager[448102]: time=2026-03-25T19:42:05.618Z level=INFO source=cluster.go:241 msg="using TLS for gossip" component=cluster
Mar 25 19:42:05 prometheus-04 prometheus-alertmanager[448102]: time=2026-03-25T19:42:05.642Z level=WARN source=cluster.go:262 msg="failed to join cluster" component=cluster err="1 error occurred:\n\t* Failed to join 49.12.57.136:9094: failed to dial: tls: failed to verify certificate: x509: cannot validate certificate for 49.12.57.136 because it doesn't contain any IP SANs\n\n"
Mar 25 19:42:05 prometheus-04 prometheus-alertmanager[448102]: time=2026-03-25T19:42:05.642Z level=INFO source=cluster.go:264 msg="will retry joining cluster every 10s" component=cluster
Mar 25 19:42:05 prometheus-04 prometheus-alertmanager[448102]: time=2026-03-25T19:42:05.642Z level=WARN source=main.go:333 msg="unable to join gossip mesh" err="1 error occurred:\n\t* Failed to join 49.12.57.136:9094: failed to dial: tls: failed to verify certificate: x509: cannot validate certificate for 49.12.57.136 because it doesn't contain any IP SANs\n\n"
~~~

Start at cluster.go:241 and trace how the --cluster.peer value is used during TLS connection setup, comparing it with the tls_client hostname workaround described in the issue. The fix is complete when a peer configured with an FQDN validates a certificate containing that hostname without requiring an IP SAN, and the reported startup failure no longer occurs.