When the nodes leave and start at the same time in a large-capacity consul cluster, some member states will continue to keep leaving, not alive.
- Dominant language
- Go
- Stars
- 6.1k
- Forks
- 609
- Avg merge
- 15h 1m
- Merged PRs (30d)
- 3
Description
I am in a consul scenario with 3 servers and 150 clients.
When 150 clients leave at the same time and restart, some member states show leaving
Further analysis, the following scenarios appear
- Node A and Node B are both rebooted
- After node A starts, it receives the leave message from B, and A caches the type and ltime of the message to recentIntents
- When A receives the join message from B, the join message is smaller than the leave message ltime. As a result, In A consul members, the state of B is always leaving.
The source code is as follows
https://github.com/hashicorp/serf/blob/830be12c38e509faa64a2a650629e91c8930a040/serf/serf.go#L1111
https://github.com/hashicorp/serf/blob/830be12c38e509faa64a2a650629e91c8930a040/serf/serf.go#L1220
https://github.com/hashicorp/serf/blob/830be12c38e509faa64a2a650629e91c8930a040/serf/serf.go#L956
https://github.com/hashicorp/serf/blob/830be12c38e509faa64a2a650629e91c8930a040/serf/serf.go#L1225
/kind bug
Contributor guide
Assessment
This issue has not been assessed yet.