cockroachdb / cockroachdb/cockroach
allocator: prevent multiple replicas from sharing a host
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
On high-CPU machines, to avoid running CRDB processes across NUMA sockets, customers want to run multiple incarnations of CRDB on a single machine. This raises the question of fault tolerance: since a machine is a failure domain, in general we never want two replicas to reside on two nodes that share an underlying machine.
By using locality tags (for example `region=x,...,host=X`) the allocator "usually" will avoid placing two replicas on nodes with matching `host=X` tag, but it's still possible that it sometimes may end up doing that, as locality diversity is strictly speaking "best effort".
We have an existing check that can prevent a new replica to be allocated on a node that has an existing replica,
https://github.com/cockroachdb/cockroach/blob/a1ef101368f567190c4b49561330ea6b4a0f5782/pkg/kv/kvserver/allocator/allocatorimpl/allocator_scorer.go#L1207-L1211
but this does not apply to the case in which multiple nodes run on a single machine (as each would have their own NodeID, and the allocator wouldn't realize they're on the same node).
[Slack](https://cockroachlabs.slack.com/archives/C0KB9Q03D/p1752788949654439)
Epic CRDB-56265
Jira issue: CRDB-54642
Contributor guide
Assessment
This issue has not been assessed yet.