apple / apple/foundationdb

Confirm before switching to an incompatible redundancy mode

Open
#4,126 0 comments 0 reactions 1 assignee Claimed by @sfc-gh-etschannen View on GitHub
operations
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

We had a testing cluster accidentally running in `single` redundancy and changed it to align to the desired redundancy of `three_data_hall`. An operator made this change and the database was rendered unavailable with the message:

```Need at least 4 log servers across unique zones, 1 proxies and 1 resolvers.```

I would've said to switch back to `single` until you add enough nodes to satisfy the configuration, but I was not around at the time. The operator did eventually add more nodes and the database became available again and there was no data loss.

Are there any protection mechanisms to prevent this from happening? It seems like there is enough information to tell the operator that same message before you actually execute the reconfiguration.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.