cockroachdb / cockroachdb/cockroach

kv,obs: persist lease history beyond 5 changes in the rangelog contained in debug.zip

Open
#124,265 0 comments 0 reactions 0 assignees View on GitHub
A-kv A-kv-observability C-enhancement O-support P-3 T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Very often, operator's are tasked with identifying the cause for an issue in the past. Knowing where particular ranges held their lease is very valuable. For example, missing timeseries metrics can be explained by a network issue on the leaseholder for the timeseries range. Another example is a cluster struggling because the lease for the liveness table was held on a bad node.

Today, we persist 5 entries for the history in our logs, and those 5 entries can constantly change and we have no historical record to answer this. The following [thread](https://cockroachlabs.slack.com/archives/C0KB9Q03D/p1715617685630799) (internal) explains why it is 5 today, and an explanation on the impact if we were to increase it.

Jira issue: CRDB-38799

Contributor guide

Open the contributing guide

Research direction

Start by locating the lease-history and rangelog implementation that produces debug.zip, then read how the current five-entry history is persisted and replaced. Trace the related observability or KV tests if present; done means debug.zip retains lease history beyond five changes without unacceptable impact described in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, distributed-systems, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.