Handle SIGTERM on CouchDB Cluster
- Dominant language
- Erlang
- Stars
- 7k
- Forks
- 1.1k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 9
Description
I am stress testing for a high availability environment of a Couch cluster on a Kubernetes cluster. The issue I am currently seeing is that when a Couch cluster node (kubernetes pod) is removed, in the time it is being removed and recreated, a series of 5xx errors are thrown for requests that are alive in that interval. Going a little deeper, I am trying to understand if Couch handles some mechanism for detecting the SIGTERM signal from the main process, in order to divert the live connections in the node that is going to be recreated to other nodes in the cluster and avoid receiving 5xx errors.
Is there a mechanism to handle SIGTERM signals in a Couch cluster? If not, it would be an interesting feature that would allow us to have more control to avoid losing connections at the time of node loss.
Contributor guide
Research direction
Start by reading the CouchDB cluster and Kubernetes shutdown behavior relevant to SIGTERM, then reproduce removal and recreation of a node while requests remain active. Determine whether live connections can be diverted to other nodes and define done as avoiding the reported 5xx errors during node loss.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- erlang, kubernetes
- Domain
- cloud, databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100