etcd-io / etcd-io/website

Add details of etcd quorum membership and (AZ's) when a node falls out

Open
#779 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
194
Forks
354
Avg merge
9d 12h
Merged PRs (30d)
3

Description

I have many customers who are smart. However, when discussing etcd and quorum they forget to realize that once the quorum membership is set, it doesn't change when a member goes offline/down.

This results in inaccurate discussions regarding 5 members instead of 3. i.e. When talking about HA etcd in 2 AZ's a customer says "well, let's go to 5 members, then I will always have quorum if one goes down."

Somewhere, 2 details fail in this scenario.
1) The Customer only has 2 AZ's, so the plan needs to account for the failure of an AZ not for individual hosts.
When an AZ goes offline either lose 2 or 3 etcd members are lost.
Planning for worst case the discussion should cover the loss of 3 members; 2/5 members are active which is less than 50% and etcd goes read-only"
2) Quorum membership does not automatically change when members go offline/down.
In the situation of 2 AZ's and 5 etcd members, when an AZ is lost either 2 or 3 etcd members go down. Discussing the loss of 3 members, the conversation I hear from customers is: "I still have 2 members, so 2/3 is more than 50%". Somehow the customer forgets that the quorum is size 5 and does not change to 3 unless manually told to do so and in their example they have 2/5 members active which is less than 50% and etcd goes read-only"

My ask is this. Either explicitly document the behavior of a 5 member quorum, or add extra text clarifying the quorum size not changing automatically during an outage.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.