Consul forwards DNS queries for type `DS` inside consul. zone to recursors
- Dominant language
- Go
- Stars
- 30.1k
- Forks
- 4.6k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 43
Description
#### Overview of the Issue
When consul is configured with a recursor, DNS queries for unrecognized record types are forwarded to the configured recursors. This forwarding also happens when resolving names in the `consul.` zone with types other than `SOA`, `NS`, `ANY`, `A`, `AAAA`, `TXT`.
In a particular setup, when dnsmasq is configured to forward `consul.` to a consul DNS server, and consul is configured to use this dnsmasq server as a recursor, this results in a DNS recursion loop. The dnsmasq configuration is described on hashicorp learn: https://learn.hashicorp.com/consul/security-networking/forwarding#dnsmasq-setup
The documentation states that consul only forwards DNS queries for names outside the `consul.` zone: https://www.consul.io/docs/agent/options.html#recursors
My expectations would also be that consul acts as an authoritative DNS server for the `consul.` zone, and does not forward it to recursors.
#### Reproduction Steps
A docker setup to reproduce this issue is available in a gist: https://gist.github.com/vierbergenlars/d5877cf8bb076fb5789f47d1ad7039fb
Alternatively:
1. Set up a consul node (or a cluster, it does not matter) with `-recursor` pointing to dnsmasq.
1. Set up dnsmasq, forwarding the `consul.` zone to the consul DNS server.
1. Perform a DNS lookup for an unsupported record type to `consul.service.consul` against either dnsmasq or consul. I used the dnssec `DS` type. `dig @127.0.0.1 -p 8600` DS consul.service.consul`
1. Even long after `dig` has finished, DNS requests keep bouncing around between consul and dnsmasq.
### Consul info for Server
Server info
```
agent:
check_monitors = 0
check_ttls = 0
checks = 0
services = 0
build:
prerelease =
revision = a82e6a7f
version = 1.5.2
consul:
acl = disabled
bootstrap = true
known_datacenters = 1
leader = true
leader_addr = 127.0.0.1:8300
server = true
raft:
applied_index = 71
commit_index = 71
fsm_pending = 0
last_contact = 0
last_log_index = 71
last_log_term = 2
last_snapshot_index = 0
last_snapshot_term = 0
latest_configuration = [{Suffrage:Voter ID:a1a793cb-2564-e6ab-8c85-6bc4a3ee35b1 Address:127.0.0.1:8300}]
latest_configuration_index = 1
num_peers = 0
protocol_version = 3
protocol_version_max = 3
protocol_version_min = 0
snapshot_version_max = 1
snapshot_version_min = 0
state = Leader
term = 2
runtime:
arch = amd64
cpu_count = 4
goroutines = 82
max_procs = 4
os = linux
version = go1.12.1
serf_lan:
coordinate_resets = 0
encrypted = false
event_queue = 1
event_time = 2
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 1
members = 1
query_queue = 0
query_time = 1
serf_wan:
coordinate_resets = 0
encrypted = false
event_queue = 0
event_time = 1
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 1
members = 1
query_queue = 0
query_time = 1
```
### Log Fragments
From the consul server:
```
2019/07/20 16:08:42 [ERR] dns: recurse failed: read udp 10.33.10.2:34942->10.33.10.3:53: i/o timeout
2019/07/20 16:08:42 [ERR] dns: all resolvers failed for {consul.service.consul. 43 1} from client 10.33.10.3:571 (udp)
2019/07/20 16:08:42 [DEBUG] dns: request for {consul.service.consul. 43 1} (udp) (2.000994556s) from client 10.33.10.3:571 (udp)
2019/07/20 16:08:42 [ERR] dns: recurse failed: read udp 10.33.10.2:59383->10.33.10.3:53: i/o timeout
2019/07/20 16:08:42 [ERR] dns: all resolvers failed for {consul.service.consul. 43 1} from client 10.33.10.3:46076 (udp)
2019/07/20 16:08:42 [ERR] dns: recurse failed: read udp 10.33.10.2:36895->10.33.10.3:53: i/o timeout
2019/07/20 16:08:42 [ERR] dns: all resolvers failed for {consul.service.consul. 43 1} from client 10.33.10.3:33552 (udp)
2019/07/20 16:08:42 [DEBUG] dns: request for {consul.service.consul. 43 1} (udp) (2.001374786s) from client 10.33.10.3:46076 (udp)
2019/07/20 16:08:42 [DEBUG] dns: request for {consul.service.consul. 43 1} (udp) (2.002385645s) from client 10.33.10.3:33552 (udp)
```
From dnsmasq:
```
dnsmasq: Maximum number of concurrent DNS queries reached (max: 150)
dnsmasq: Maximum number of concurrent DNS queries reached (max: 150)
```
Contributor guide
Assessment
This issue has not been assessed yet.