cockroachdb / cockroachdb/cockroach-operator

[Regression] Decommission is broken

Open
#542 18 comments 0 reactions 1 assignee Claimed by @alinadonisa View on GitHub
bug
Dominant language
Go
Stars
318
Forks
104
Avg merge
1d 6h
Merged PRs (30d)
1

Description

The statefulset is stopping the pod before `cockroach node decommission` is being executed, so the node is showing up as failed instead of as decommissioned.

The PVC also gets deleted so there is no way to recover from this state if the scale down caused a loss of quorum because the data gets destroyed.

Steps to reproduce:
create 4 node cluster
change node count to 3 nodes
![Screen Shot 2021-05-28 at 12 26 31 PM](https://user-images.githubusercontent.com/4730669/120014337-efe6e280-bfaf-11eb-94c6-a3273e3210e2.png)
[decommission-regression.log](https://github.com/cockroachdb/cockroach-operator/files/6562142/decommission-regression.log)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.