hashicorp / hashicorp/consul

Watches configured in the agent config are incompatible with Agent TLS when non-localhost address is used for client address

Open
#11,683 1 comment 0 reactions 0 assignees View on GitHub
type/bug
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
1d 18h
Merged PRs (30d)
39

Description

This bug was originally reported in #11357. Note this only applies to the [watches in the agent config](https://www.consul.io/docs/agent/options#watches). Watches [that use the CLI commad](https://www.consul.io/commands/watch) should work because the user can specify a hostname instead of an IP address.

The problem can be seen following this code path:
https://github.com/hashicorp/consul/blob/7bbad6fe560ab27f51dd6c5cd16f28418082424c/agent/config/runtime.go#L1712-L1715

https://github.com/hashicorp/consul/blob/7bbad6fe560ab27f51dd6c5cd16f28418082424c/agent/config/runtime.go#L1594-L1595

The problem is that the configured watches look up the address to dial from the agent configuration. The agent configuration must use IP address to bind, but the TLS certificates will not necessarily contain IP SANs, because IP addresses could change. They generally will contain DNS SANs, or IP SANs for `localhost 127.0.0.1`.

When localhost is used as the client address the watches work fine because the TLS certificate has those listed as IP SANs, but as soon as some interface is used for the client address, it no longer works.

**Workarounds**:
* use a localhost address as the first client addresses (**recommended**) - this is a common setup, which is probably why we have not noticed this problem before. Each of these https://www.consul.io/docs/agent/options#addresses supports a space separated list, so use `127.0.0.1 `.
* disable `verify_outgoing` - this is particularly not great because it also disables it for all RPC communication, which is not recommended

One option for fixing this problem would be to allow the user to configure a `hostname` to use for performing these watches. That way the TLS certificate would be valid. However it would require the user to make sure their DNS resolves that address correctly (possibly by using `/etc/hosts`).

Contributor guide

Open the contributing guide

Research direction

Start in agent/config/runtime.go at the cited lines around 1712-1715 and 1594-1595, tracing how configured watches choose their dial address and use Agent TLS. Reproduce the failure with a non-localhost client address and outgoing verification enabled; done means configured watches work with TLS without disabling verification.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
networking, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.