cockroachdb / cockroachdb/cockroach
kvserver: investigate and document non-cooperative lease acquisition timeline
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
The timeline for non-cooperative lease acquisitions, and thus the unavailability distribution for various failure modes, can be hard to reason about due to the many factors involved. We should investigate and document this, and compare it to empirical unavailability observations (see #103654). This involves e.g.:
* Lease duration
* Lease renewal intervals
* Raft election timeouts (randomized)
* Raft heartbeat intervals
* Raft reproposal timeouts (for lease proposals)
* RPC heartbeat timeouts
* RPC circuit breaker policy
* gRPC keepalive and network timeouts
* TCP network and connection timeouts
* DistSender cache invalidation
* DistSender replica ordering and retry policy
* And probably more
There's a basic, incomplete timeline here:
https://github.com/cockroachdb/cockroach/blob/735cf2103246194a98899008ec9d80b110b083e5/pkg/base/config.go#L171-L182
Jira issue: CRDB-28114
Contributor guide
Research direction
Start with the incomplete timeline in pkg/base/config.go around lines 171-182, then trace the listed lease, Raft, RPC, gRPC, TCP, and DistSender factors. Compare the resulting non-cooperative lease-acquisition timeline with the empirical unavailability observations in issue #103654; done means documenting the timeline and failure-mode distribution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, grpc
- Domain
- databases, distributed-systems
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100