cockroachdb / cockroachdb/cockroach
kvserver: dump traces for failed node liveness heartbeats
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Describe the problem**
Currently, we don't have great observability into why NodeLiveness heartbeats failed. We should consider tracing liveness heartbeats and dumping these traces (both on the client and on the server) any time a liveness heartbeat fails or times out. s/o @stevendanna for the suggestion.
Jira issue: CRDB-54844
Contributor guide
Research direction
Start by locating the Go implementation of NodeLiveness heartbeats and the existing client- and server-side tracing or trace-dump mechanisms. Trace the failure and timeout paths on both sides and determine how existing tests cover them. Done means failed or timed-out liveness heartbeats produce the requested traces for both the client and server.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100