cockroachdb / cockroachdb/cockroach

allocator: take lease preferences into account for lease convergence

Aperta
#93,258 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
A-kv-distribution C-bug T-kv
Lingua principale
Go
Stelle
32.5k
Fork
4.1k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

When load-based lease rebalancing is disabled, the allocator falls back to rebalancing leases by count. It does this by calculating the mean lease count across the cluster, and then transfer a lease from a node with `leases > mean` to one with `leases < mean`. However, this does not take lease preferences into account, and the mean calculation will include the 0 lease counts from nodes that are not within the lease preferences.

Consider an example with three nodes in each of `asia`, `eu`, `us`, and a lease preference set to `us`:

* `asia1`: 0
* `asia2`: 0
* `asia3`: 0
* `eu1`: 0
* `eu2`: 0
* `eu3`: 0
* `us1`: 133
* `us2`: 133
* `us3`: 33

In this case, the operator might expect the leases to rebalance such that all `us` nodes have ~100 leases each, but this won't happen. `us3` will be stuck at 33 leases, because the average lease count is 300 / 9 = 33 and it's therefore at the mean and thus not considered underfull.

The lease convergence calculation needs to take lease preferences into account, as well as which preferences can actually be satisfied (e.g. in the case of unavailable nodes/regions).

This determination is made here:

https://github.com/cockroachdb/cockroach/blob/d4a2c743e02c82a903fdbac782f1d5df5e8c33a1/pkg/kv/kvserver/allocator.go#L2170-L2186

Based on the lease candidate store mean which is calculated here:

https://github.com/cockroachdb/cockroach/blob/0f816899cd3e8605b3d8955967a8956b4cf15fb9/pkg/kv/kvserver/store_pool.go#L778

Jira issue: CRDB-22249

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.