cockroachdb / cockroachdb/cockroach

Critical node endpoint always returns incorrect response for 3-node clusters

Open
#105,238 0 comments 0 reactions 0 assignees View on GitHub
C-bug T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

The [critical node endpoint ](https://www.cockroachlabs.com/docs/v23.1/monitoring-and-alerting.html#critical-nodes-endpoint) is returning an incorrect response.

Repro:

1. start demo with cockroach.exe demo --no-example-database --insecure --nodes 3
2. \demo shutdown 3
3. post critical nodes
4. \demo restart 3
5. post critical nodes (observe critical nodes are still present after a long period of time)

[Slack context and repro](https://cockroachlabs.slack.com/archives/C0159JK877C/p1687289339344069)

[Related issue
](https://github.com/cockroachdb/cockroach/issues/103910)

This results in false positives. The critical nodes endpoint always reports that all nodes in a 3-node cluster are critical, because the default replication factor for system ranges is 5 (though elsewhere CockroachDB makes allowances that a 3-node cluster only has a effective replication factor of 3).

Jira issue: CRDB-28932

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.