hashicorp / hashicorp/consul

Node names should be case insensitive

Open
#1,911 5 comments 0 reactions 0 assignees View on GitHub
needs-discussion theme/operator-usability type/bug
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
1d 18h
Merged PRs (30d)
39

Description

Currently node names are case sensitive, which can lead to node flapping if two nodes have the same name and one of them fails/leaves. For instance, this can be reproduced by having the same node re-join with its hostname changed to uppercase. Similarly node flapping occurs if a different node joins with some variation in casing from an existing host.

According to https://tools.ietf.org/html/rfc4343, DNS treats hostnames as case insensitive, and since the node name indirectly affects queries to Consul's DNS, the same should apply. For instance, if the serf health for that node flaps, so will queries to its services.

Case where the node is marked as failed:

```
# consul members
consul members
Node Address Status Type Build Protocol DC
9a6e233f88cd 172.17.0.2:8301 alive server 0.6.3 2 dc1
TEST 172.17.0.3:8301 alive client 0.6.3 2 dc1
test 172.17.0.3:8301 failed client 0.6.3 2 dc1
```

```
2016/04/01 18:05:32 [INFO] consul: member 'test' failed, marking health critical
2016/04/01 18:05:32 [INFO] consul: member 'TEST' joined, marking health alive
2016/04/01 18:06:32 [INFO] consul: member 'test' failed, marking health critical
2016/04/01 18:06:32 [INFO] consul: member 'TEST' joined, marking health alive
2016/04/01 18:06:37 [INFO] serf: attempting reconnect to test 172.17.0.3:8301
2016/04/01 18:06:51 [INFO] agent.rpc: Accepted client: 127.0.0.1:45471
2016/04/01 18:06:57 [INFO] agent.rpc: Accepted client: 127.0.0.1:45472
2016/04/01 18:07:32 [INFO] consul: member 'test' failed, marking health critical
2016/04/01 18:07:32 [INFO] consul: member 'TEST' joined, marking health alive
2016/04/01 18:08:07 [INFO] serf: attempting reconnect to test 172.17.0.3:8301
2016/04/01 18:08:32 [INFO] consul: member 'test' failed, marking health critical
2016/04/01 18:08:32 [INFO] consul: member 'TEST' joined, marking health alive
2016/04/01 18:09:07 [INFO] serf: attempting reconnect to test 172.17.0.3:8301
2016/04/01 18:09:32 [INFO] consul: member 'test' failed, marking health critical
2016/04/01 18:09:32 [INFO] consul: member 'TEST' joined, marking health alive

```

Case where the node is marked as left:

```
# consul members
Node Address Status Type Build Protocol DC
9a6e233f88cd 172.17.0.2:8301 alive server 0.6.3 2 dc1
TEST 172.17.0.3:8301 alive client 0.6.3 2 dc1
test 172.17.0.3:8301 left client 0.6.3 2 dc1
```

```
2016/04/01 18:12:08 [INFO] Force leaving node: test
2016/04/01 18:12:08 [INFO] serf: EventMemberLeave (forced): test 172.17.0.3
2016/04/01 18:12:08 [INFO] consul: member 'test' left, deregistering
2016/04/01 18:12:10 [INFO] agent.rpc: Accepted client: 127.0.0.1:45500
2016/04/01 18:12:17 [INFO] agent.rpc: Accepted client: 127.0.0.1:45501
2016/04/01 18:12:19 [INFO] agent.rpc: Accepted client: 127.0.0.1:45502
2016/04/01 18:12:31 [INFO] agent.rpc: Accepted client: 127.0.0.1:45503
2016/04/01 18:12:32 [INFO] consul: member 'TEST' joined, marking health alive
2016/04/01 18:12:39 [INFO] agent.rpc: Accepted client: 127.0.0.1:45506
2016/04/01 18:12:45 [INFO] agent.rpc: Accepted client: 127.0.0.1:45507
2016/04/01 18:13:32 [INFO] consul: member 'test' left, deregistering
2016/04/01 18:13:32 [INFO] consul: member 'TEST' joined, marking health alive
2016/04/01 18:14:29 [INFO] agent.rpc: Accepted client: 127.0.0.1:45514
2016/04/01 18:14:32 [INFO] consul: member 'test' left, deregistering
2016/04/01 18:14:32 [INFO] consul: member 'TEST' joined, marking health alive
2016/04/01 18:15:21 [INFO] agent.rpc: Accepted client: 127.0.0.1:45519
2016/04/01 18:15:32 [INFO] consul: member 'test' left, deregistering
2016/04/01 18:15:32 [INFO] consul: member 'TEST' joined, marking health alive
```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported behavior with the Consul members command and a node that rejoins with a hostname whose casing changed. Trace the Serf membership and Consul DNS paths involved in marking nodes failed or left, then verify that casing variations no longer create separate node entries or cause service-query flapping.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.