cockroachdb / cockroachdb/cockroach

liveness: distinguish between cases where ErrEpochCondFailed is returned

Open
#125,093 0 comments 0 reactions 0 assignees View on GitHub
C-enhancement O-support P-3 T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

A node may attempt to, but fail to, increment another node's liveness record for two reasons:

1. It has received a gossip update with fresher node liveness information compared to what it looked at when it decided to increment another node's liveness epoch. All this happens on the local node.
2. It attempts to increment a node's liveness epoch, but the node is able to heartbeat before the increment can go through. This happens on the leaseholder of the node liveness range.

See:

https://github.com/cockroachdb/cockroach/blob/4425d3a8a4579611cd271774825f3c343893d9e9/pkg/kv/kvserver/liveness/liveness.go#L1136-L1141

As both these return ErrEpochCondFailed, both of them will have log lines of the form:

```
failed to increment leaseholder's epoch: liveness record changed while incrementing epoch for liveness
```

It's often useful to distinguish between these two cases; we should.

Jira issue: CRDB-39230

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.