cockroachdb / cockroachdb/cockroach

Sentry: replica.go:1736: log.Fatal: read ×: no such device or address (1) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).assertStateRaftMuLocke...

Open
#157,910 1 comment 0 reactions 0 assignees View on GitHub
branch-release-24.3 C-bug O-sentry T-kv X-blathers-triaged
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: [https://cockroach-labs.sentry.io/issues/7034660648/?referrer=webhooks_plugin](https://cockroach-labs.sentry.io/issues/7034660648/?referrer=webhooks_plugin)

Panic Message:

```
replica.go:1736: log.Fatal: read ×: no such device or address
(1) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).assertStateRaftMuLockedReplicaMuRLocked
| pkg/kv/kvserver/replica.go:1736
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaStateMachine).ApplySideEffects
| pkg/kv/kvserver/replica_application_state_machine.go:210
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.mapCheckedCmdIter
| pkg/kv/kvserver/apply/cmd.go:209
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).applyOneBatch
| pkg/kv/kvserver/apply/task.go:290
| github.com/cockroachdb/cockroach/pkg/kv/kvserver/apply.(*Task).ApplyCommittedEntries
| pkg/kv/kvserver/apply/task.go:246
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReadyRaftMuLocked
| pkg/kv/kvserver/replica_raft.go:1269
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReady
| pkg/kv/kvserver/replica_raft.go:846
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).processReady
| pkg/kv/kvserver/store_raft.go:682
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftSchedulerShard).worker
| pkg/kv/kvserver/scheduler.go:419
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).Start.func2
| pkg/kv/kvserver/scheduler.go:319
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
| pkg/util/stop/stopper.go:498
| runtime.goexit
| src/runtime/asm_arm64.s:1222
Wraps: (2) secondary error attachment
| read ×: no such device or address
| (1) read ×
| Wraps: (2) no such device or address
| Error types: (1) *fs.PathError (2) syscall.Errno
Wraps: (3) log.Fatal: read ×: no such device or address
Error types: (1) *withstack.withStack (2) *secondary.withSecondaryError (3) *errutil.leafError
-- report composition:
*errutil.leafError: log.Fatal: read ×: no such device or address
*secondary.withSecondaryError
replica.go:1736: *withstack.withStack (top exception)
```

Stacktrace (expand for inline code snippets):

[src/runtime/asm_arm64.s#L1221-L1223](src/runtime/asm_arm64.s#L1221-L1223)
[pkg/util/stop/stopper.go#L497-L499](pkg/util/stop/stopper.go#L497-L499)
[pkg/kv/kvserver/scheduler.go#L318-L320](pkg/kv/kvserver/scheduler.go#L318-L320)
[pkg/kv/kvserver/scheduler.go#L418-L420](pkg/kv/kvserver/scheduler.go#L418-L420)
[pkg/kv/kvserver/store_raft.go#L681-L683](pkg/kv/kvserver/store_raft.go#L681-L683)
[pkg/kv/kvserver/replica_raft.go#L845-L847](pkg/kv/kvserver/replica_raft.go#L845-L847)
[pkg/kv/kvserver/replica_raft.go#L1268-L1270](pkg/kv/kvserver/replica_raft.go#L1268-L1270)
[pkg/kv/kvserver/apply/task.go#L245-L247](pkg/kv/kvserver/apply/task.go#L245-L247)
[pkg/kv/kvserver/apply/task.go#L289-L291](pkg/kv/kvserver/apply/task.go#L289-L291)
[pkg/kv/kvserver/apply/cmd.go#L208-L210](pkg/kv/kvserver/apply/cmd.go#L208-L210)
[pkg/kv/kvserver/replica_application_state_machine.go#L209-L211](pkg/kv/kvserver/replica_application_state_machine.go#L209-L211)
[pkg/kv/kvserver/replica.go#L1735-L1737](pkg/kv/kvserver/replica.go#L1735-L1737)

```
src/runtime/asm_arm64.s in runtime.goexit at line 1222
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 498
pkg/kv/kvserver/scheduler.go in pkg/kv/kvserver.(*raftScheduler).Start.func2 at line 319
pkg/kv/kvserver/scheduler.go in pkg/kv/kvserver.(*raftSchedulerShard).worker at line 419
pkg/kv/kvserver/store_raft.go in pkg/kv/kvserver.(*Store).processReady at line 682
pkg/kv/kvserver/replica_raft.go in pkg/kv/kvserver.(*Replica).handleRaftReady at line 846
pkg/kv/kvserver/replica_raft.go in pkg/kv/kvserver.(*Replica).handleRaftReadyRaftMuLocked at line 1269
pkg/kv/kvserver/apply/task.go in pkg/kv/kvserver/apply.(*Task).ApplyCommittedEntries at line 246
pkg/kv/kvserver/apply/task.go in pkg/kv/kvserver/apply.(*Task).applyOneBatch at line 290
pkg/kv/kvserver/apply/cmd.go in pkg/kv/kvserver/apply.mapCheckedCmdIter at line 209
pkg/kv/kvserver/replica_application_state_machine.go in pkg/kv/kvserver.(*replicaStateMachine).ApplySideEffects at line 210
pkg/kv/kvserver/replica.go in pkg/kv/kvserver.(*Replica).assertStateRaftMuLockedReplicaMuRLocked at line 1736
```

### Tags

| Tag | Value |
| --- | --- |
| Command | server |
| Environment | v24.3.18 |
| Go Version | go1.22.8 X:nocoverageredesign |
| Platform | linux arm64 |
| Distribution | CCL |
| Cockroach Release | v24.3.18 |
| Cockroach SHA | 37439c410d4583708c898b13a2f9e906f444867b |
| # of CPUs | 32 |
| # of Goroutines | 3395 |

Jira issue: CRDB-57013

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.