cockroachdb / cockroachdb/cockroach
kvserver: add a gossip histogram metrics for latency
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Is your feature request related to a problem? Please describe.**
Timely gossip delivery is important for liveness and rebalancing decisions. In some cases we have either seen slow gossip propagation as a problem causing liveness failures. Having a histogram of the delay between when a gossip was first published, `OrigStamp`, and when it was added to the `infoStore` will give us insight into these delays. Additionally in cross-regional networks gossip messages often have to cross regions multiple times to get from the source to the destination. For a large cross-region cluster this could result in unacceptable delays for time-sensitive messages.
**Describe the solution you'd like**
A histogram which show the time difference between when an info is originally created vs when it was added to a store in `addInfo` would help debug complex gossip network delays. There is a separate histogram already added as part of #125148 for the callback processing delay, but we also care about the propagation delay.
Jira issue: CRDB-39458
Contributor guide
Assessment
This issue has not been assessed yet.