cockroachdb / cockroachdb/cockroach
Sentry: cache.go:147: log.Fatal: invalid new liveness record; found to be empty (1) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/kv/kvserver/liveness.(*Cache).maybeU...
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.
Sentry Link: [https://cockroach-labs.sentry.io/issues/7052458538/?referrer=webhooks_plugin](https://cockroach-labs.sentry.io/issues/7052458538/?referrer=webhooks_plugin)
Panic Message:
```
cache.go:147: log.Fatal: invalid new liveness record; found to be empty
(1) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/liveness.(*Cache).maybeUpdate
| pkg/kv/kvserver/liveness/cache.go:147
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/liveness.(*NodeLiveness).heartbeatInternal.func3
| pkg/kv/kvserver/liveness/liveness.go:835
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/liveness.(*storageImpl).Update
| pkg/kv/kvserver/liveness/storage.go:125
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/liveness.(*NodeLiveness).updateLivenessAttempt
| pkg/kv/kvserver/liveness/liveness.go:1136
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/liveness.(*NodeLiveness).updateLiveness
| pkg/kv/kvserver/liveness/liveness.go:1070
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/liveness.(*NodeLiveness).heartbeatInternal
| pkg/kv/kvserver/liveness/liveness.go:833
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/liveness.(*NodeLiveness).Start.func1.1
| pkg/kv/kvserver/liveness/liveness.go:661
| github.com/cockroachdb/cockroach/pkg/util/timeutil.RunWithTimeout
| pkg/util/timeutil/timeout.go:28
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/liveness.(*NodeLiveness).Start.func1
| pkg/kv/kvserver/liveness/liveness.go:643
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1
| pkg/util/stop/stopper.go:438
| runtime.goexit
| src/runtime/asm_amd64.s:1700
Wraps: (2) log.Fatal: invalid new liveness record; found to be empty
Error types: (1) *withstack.withStack (2) *errutil.leafError
-- report composition:
*errutil.leafError: log.Fatal: invalid new liveness record; found to be empty
cache.go:147: *withstack.withStack (top exception)
```
Stacktrace (expand for inline code snippets):
[src/runtime/asm_amd64.s#L1699-L1701](src/runtime/asm_amd64.s#L1699-L1701)
[pkg/util/stop/stopper.go#L437-L439](pkg/util/stop/stopper.go#L437-L439)
[pkg/kv/kvserver/liveness/liveness.go#L642-L644](pkg/kv/kvserver/liveness/liveness.go#L642-L644)
[pkg/util/timeutil/timeout.go#L27-L29](pkg/util/timeutil/timeout.go#L27-L29)
[pkg/kv/kvserver/liveness/liveness.go#L660-L662](pkg/kv/kvserver/liveness/liveness.go#L660-L662)
[pkg/kv/kvserver/liveness/liveness.go#L832-L834](pkg/kv/kvserver/liveness/liveness.go#L832-L834)
[pkg/kv/kvserver/liveness/liveness.go#L1069-L1071](pkg/kv/kvserver/liveness/liveness.go#L1069-L1071)
[pkg/kv/kvserver/liveness/liveness.go#L1135-L1137](pkg/kv/kvserver/liveness/liveness.go#L1135-L1137)
[pkg/kv/kvserver/liveness/storage.go#L124-L126](pkg/kv/kvserver/liveness/storage.go#L124-L126)
[pkg/kv/kvserver/liveness/liveness.go#L834-L836](pkg/kv/kvserver/liveness/liveness.go#L834-L836)
[pkg/kv/kvserver/liveness/cache.go#L146-L148](pkg/kv/kvserver/liveness/cache.go#L146-L148)
```
src/runtime/asm_amd64.s in runtime.goexit at line 1700
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1 at line 438
pkg/kv/kvserver/liveness/liveness.go in pkg/kv/kvserver/liveness.(*NodeLiveness).Start.func1 at line 643
pkg/util/timeutil/timeout.go in pkg/util/timeutil.RunWithTimeout at line 28
pkg/kv/kvserver/liveness/liveness.go in pkg/kv/kvserver/liveness.(*NodeLiveness).Start.func1.1 at line 661
pkg/kv/kvserver/liveness/liveness.go in pkg/kv/kvserver/liveness.(*NodeLiveness).heartbeatInternal at line 833
pkg/kv/kvserver/liveness/liveness.go in pkg/kv/kvserver/liveness.(*NodeLiveness).updateLiveness at line 1070
pkg/kv/kvserver/liveness/liveness.go in pkg/kv/kvserver/liveness.(*NodeLiveness).updateLivenessAttempt at line 1136
pkg/kv/kvserver/liveness/storage.go in pkg/kv/kvserver/liveness.(*storageImpl).Update at line 125
pkg/kv/kvserver/liveness/liveness.go in pkg/kv/kvserver/liveness.(*NodeLiveness).heartbeatInternal.func3 at line 835
pkg/kv/kvserver/liveness/cache.go in pkg/kv/kvserver/liveness.(*Cache).maybeUpdate at line 147
```
### Tags
| Tag | Value |
| --- | --- |
| Command | server |
| Environment | v25.3.4 |
| Go Version | go1.23.12 X:nocoverageredesign |
| Platform | linux amd64 |
| Distribution | CCL |
| Cockroach Release | v25.3.4 |
| Cockroach SHA | 1cddb3a22695e967197844b214115eb645473efe |
| # of CPUs | 4 |
| # of Goroutines | 355 |
Jira issue: CRDB-57169
Contributor guide
Assessment
This issue has not been assessed yet.