hashicorp / hashicorp/serf

member-join event does not fire after resume if there are also updates to recieve

Open
#393 0 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Go
Stars
6.1k
Forks
609
Avg merge
15h 1m
Merged PRs (30d)
3

Description

If a node resumes connection with the cluster at the same time it is getting tags set the client serf agent does not execute the member join handler.

**For example:**

Node A: `nohup serf agent &`
Node B: `nohup serf agent &`
Node A: `serf join "ip address for node B"`
Node B: Create a processes that continually updates tags (tag-updater.sh)

``` bash
#!/usr/bin/env bash
for i in $(seq 500); do
serf tags -set whipple=$i
sleep 0.5
done
```

Node A: `serf monitor`
**Prepare to watch updates come in**

Node B: `bash ./tag-updater.sh`

Yank network connection on Node B
Wait for node A serf connection to register node B as failed
Reconnect network for Node B
Watch Node A as the member-join event is skipped and member-update is executed instead.

```
2016/05/27 19:44:27 [INFO] serf: EventMemberFailed: DD:CC:77:55:CC:99 xx.xx.xx.xx
2016/05/27 19:44:28 [INFO] agent: Received event: member-update
2016/05/27 19:44:28 [INFO] agent: Received event: member-failed
...
2016/05/27 19:44:45 [INFO] serf: EventMemberJoin: DD:CC:77:55:CC:99
2016/05/27 19:44:46 [INFO] serf: EventMemberUpdate: DD:CC:77:55:CC:99
2016/05/27 19:44:46 [INFO] serf: EventMemberUpdate: DD:CC:77:55:CC:99
2016/05/27 19:44:47 [INFO] serf: EventMemberUpdate: DD:CC:77:55:CC:99
2016/05/27 19:44:47 [INFO] agent: Received event: member-update
```

Is this expected behavior, or a misunderstanding about what events are guaranteed to happen after a member-fail event?

Thank you,

-wjh

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.