cockroachdb / cockroachdb/cockroach
closedts: understand spurious "closed timestamps side-transport connection dropped" errors
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Across our fleet, we see[^1] a significant volume of these logs:
> closed timestamps side-transport connection dropped from node: 9 (grpc: ‹×› [code 1/Canceled])
These are likely spurious, and not caused by network issues.
We saw these errors in an escalation as well, during periods of time in which no nodes were restarting (which might otherwise be a likely source of those errors).
[^1]: https://us5.datadoghq.com/logs?query=service%3Acockroachdb%20%40channel%3A%28DEV%29%20%22closed%20timestamps%20side-transport%20connection%20dropped%22&agg_m=count&agg_m_source=base&agg_t=count&cols=host%2Cservice&fromUser=true&messageDisplay=inline&refresh_mode=sliding&storage=flex_tier&stream_sort=desc&viz=stream&from_ts=1759133960972&to_ts=1759134260972&live=true
Jira issue: CRDB-54887
Contributor guide
Assessment
This issue has not been assessed yet.