hashicorp / hashicorp/consul

DNS interface always returns node metadata

Open
#6,567 2 comments 0 reactions 0 assignees View on GitHub
theme/dns
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
2d 6h
Merged PRs (30d)
43

Description

#### Overview of the Issue

We're using consul configuration when one consul service is destination of another one like alias. Our destination consul service has several nodes. There is a problem when client try to resolve high-level consul service through Microsoft DNS recursor. It seems to me cuz consul returns TXT records with node metadata in answer section of DNS answer. I have tried the consul option `enable_additional_node_meta_txt = false` but it doesn't help. As I can see this option has to disable all metadata from any DNS answers.

This problem started after consul upgrade from 0.8.5 to 1.6.0

#### Reproduction Steps

High-Level service

[
{
"ID": "",
"Node": "proxysql-cluster-a",
"Address": "proxysql-cluster-a.service.consul.example.com",
"Datacenter": "qa",
"TaggedAddresses": null,
"NodeMeta": null,
"ServiceKind": "",
"ServiceID": "proxysql",
"ServiceName": "proxysql",
"ServiceTags": [],
"ServiceAddress": "",
"ServiceWeights": {
"Passing": 1,
"Warning": 1
},
"ServiceMeta": {},
"ServicePort": 0,
"ServiceEnableTagOverride": false,
"ServiceProxy": {
"MeshGateway": {}
},
"ServiceConnect": {},
"CreateIndex": 6463551881,
"ModifyIndex": 6463551881
}
]

Destination service

[
{
"ID": "e2b33c78-3621-f838-056c-844b13691aaa",
"Node": "proxysqla001.example.com",
"Address": "10.101.38.29",
"Datacenter": "qa",
"TaggedAddresses": {
"lan": "10.101.38.29",
"wan": "10.101.38.29"
},
"NodeMeta": {
"machine_model": "oVirt Node",
"memory_total": "16641347584 bytes",
"processors_count": "8 cores"
},
"ServiceKind": "",
"ServiceID": "proxysql_consul",
"ServiceName": "proxysql-cluster-a"",
"ServiceTags": [],
"ServiceAddress": "",
"ServiceWeights": {
"Passing": 1,
"Warning": 1
},
"ServiceMeta": {},
"ServicePort": 6033,
"ServiceEnableTagOverride": false,
"ServiceProxy": {
"MeshGateway": {}
},
"ServiceConnect": {},
"CreateIndex": 6463551881,
"ModifyIndex": 6463551881
},
{
"ID": "12faa91d-8a92-56b4-bc66-6a6d0ef07b49",
"Node": "proxysqla002.example.com",
"Address": "10.106.50.65",
"Datacenter": "qa",
"TaggedAddresses": {
"lan": "10.106.50.65",
"wan": "10.106.50.65"
},
"NodeMeta": {
"machine_model": "oVirt Node",
"memory_total": "16641347584 bytes",
"processors_count": "8 cores"
},
"ServiceKind": "",
"ServiceID": "proxysql_consul",
"ServiceName": "proxysql-cluster-a"",
"ServiceTags": [],
"ServiceAddress": "",
"ServiceWeights": {
"Passing": 1,
"Warning": 1
},
"ServiceMeta": {},
"ServicePort": 6033,
"ServiceEnableTagOverride": false,
"ServiceProxy": {
"MeshGateway": {}
},
"ServiceConnect": {},
"CreateIndex": 6463551881,
"ModifyIndex": 6463551881
},
{
"ID": "cc67c839-76cd-0769-ebc2-291c85891681",
"Node": "proxysqla003.example.com",
"Address": "10.110.50.68",
"Datacenter": "qa",
"TaggedAddresses": {
"lan": "10.110.50.68",
"wan": "10.110.50.68"
},
"NodeMeta": {
"machine_model": "oVirt Node",
"memory_total": "16641347584 bytes",
"processors_count": "8 cores"
},
"ServiceKind": "",
"ServiceID": "proxysql_consul",
"ServiceName": "proxysql-cluster-a"",
"ServiceTags": [],
"ServiceAddress": "",
"ServiceWeights": {
"Passing": 1,
"Warning": 1
},
"ServiceMeta": {},
"ServicePort": 6033,
"ServiceEnableTagOverride": false,
"ServiceProxy": {
"MeshGateway": {}
},
"ServiceConnect": {},
"CreateIndex": 6463551881,
"ModifyIndex": 6463551881
}
]

### ConsulConfiguration:

```
ConsulService -> proxysql.service.consul.example.com -> Node: proxysqla

Node: proxysqla -> ConsulService: proxysql-cluster-a.service.consul.example.com.

ConsulService: proxysql-cluster-a.service.consul.example.com. -> Node: proxysqla00[1-3].example.com
```

### DNS interface:
```
host proxysql.service.consul.example.com
proxysql.service.consul.example.com is an alias for proxysql-cluster-a.service.consul.example.com.
proxysql-cluster-a.service.consul.example.com has address 10.106.50.65
proxysql-cluster-a.service.consul.example.com has address 10.101.38.29
proxysql-cluster-a.service.consul.example.com has address 10.110.50.68
```

### Request to destination consul service directly
```
dig @qa-consul007.example.com proxysql-cluster-a.service.consul.example.com

; <<>> DiG 9.10.6 <<>> @qa-consul007.example.com proxysql-cluster-a.service.consul.example.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49255
;; flags: qr aa rd; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;proxysql-cluster-a.service.consul.example.com. IN A

;; ANSWER SECTION:
proxysql-cluster-a.service.consul.example.com. 0 IN A 10.106.50.65
proxysql-cluster-a.service.consul.example.com. 0 IN A 10.101.38.29
proxysql-cluster-a.service.consul.example.com. 0 IN A 10.110.50.68
```

### Request to high-level consul service directly
```
dig @qa-consul007.example.com proxysql.service.consul.example.com A

; <<>> DiG 9.10.6 <<>> @qa-consul007.example.com
proxysql.service.consul.example.com A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11683
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;proxysql.service.consul.example.com. IN A

;; ANSWER SECTION:
proxysql.service.consul.example.com. 0 IN CNAME proxysql-cluster-a.service.consul.example.com.
proxysql-cluster-a.service.consul.example.com. 0 IN A 10.101.38.29
proxysql-cluster-a.service.consul.example.com. 0 IN TXT "consul-network-segment="
proxysql-cluster-a.service.consul.example.com. 0 IN A 10.106.50.65

```

### Request to high level consul service through MS DNS
The 172.25.23.5 is MS DNS recursor

```
dig @172.25.23.5 proxysql.service.consul.example.com.
; <<>> DiG 9.10.6 <<>> @172.25.23.5 proxysql.service.consul.example.com.
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 10249
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4000
;; QUESTION SECTION:
;proxysql.service.consul.example.com. IN A

;; AUTHORITY SECTION:
qa-consul.example.com. 0 IN SOA ns.qa-consul.example.com. hostmaster.qa-consul.example.com. 1570005795 3600 600 86400 0

;; Query time: 13 msec
;; SERVER: 172.25.23.5#53(172.25.23.5)
;; WHEN: Wed Oct 02 11:43:15 IDT 2019
;; MSG SIZE rcvd: 135

# host -d -t a proxysql.service.consul.example.com. 172.25.23.5
Trying "proxysql.service.consul.example.com."
Using domain server:
Name: 172.25.23.5
Address: 172.25.23.5#53
Aliases:

Host proxysql.service.consul.example.com. not found: 2(SERVFAIL)
Received 71 bytes from 172.25.23.5#53 in 14 ms
```

### Consul info Server

Server info

agent:
check_monitors = 11
check_ttls = 0
checks = 11
services = 6
build:
prerelease =
revision = 944cc710
version = 1.6.0
consul:
acl = disabled
known_servers = 5
server = false
runtime:
arch = amd64
cpu_count = 40
goroutines = 67
max_procs = 40
os = linux
version = go1.12.8
serf_lan:
coordinate_resets = 0
encrypted = true
event_queue = 0
event_time = 22
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 13142
members = 560
query_queue = 0
query_time = 13

### Operating system and Environment details

* CentOS Linux release 7.4.1708 (Core)
* Linux example.com 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
* Microsoft Windows 2016

### Log Fragments

Sep 25 15:17:25 consul007 consul: 2019/09/25 15:17:25 [DEBUG] dns: request for name proxysql-cluster-a.service.consul.example.com. type A class IN (took 213.953µs) from client 172.25.13.12:50864 (udp)
Sep 25 15:18:17 consul007 consul: 2019/09/25 15:18:17 [DEBUG] dns: request for name proxysql.service.consul.example.com. type A class IN (took 403.771µs) from client 172.25.13.12:61334 (udp)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.