Sentry: replica_consistency.go:832: log.Fatal: ATTENTION: (1) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).computeChecksumPostApply.func2 | ...

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: [https://cockroach-labs.sentry.io/issues/7218940409/?referrer=webhooks_plugin](https://cockroach-labs.sentry.io/issues/7218940409/?referrer=webhooks_plugin)

Panic Message:

```
replica_consistency.go:832: log.Fatal: ATTENTION:
(1) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).computeChecksumPostApply.func2
| pkg/kv/kvserver/replica_consistency.go:832
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
| pkg/util/stop/stopper.go:498
| runtime.goexit
| src/runtime/asm_arm64.s:1222
Wraps: (2) log.Fatal: ATTENTION:
|
| This node is terminating because a replica inconsistency was detected between [n803,s6404,r26961875/3:×]
| and its other replicas: (n687,s5244):1,(n727,s5636):2,(n803,s6404):3,(n737,s5741):4,(n695,s5320):5. Please check your cluster-wide log files for more
| information and contact the CockroachDB support team. It is not necessarily safe
| to replace this node; cluster data may still be at risk of corruption.
|
| A checkpoints directory to aid (expert) debugging should be present in:
| /data/crdb/data3/auxiliary
|
| A file preventing this node from restarting was placed at:
| /data/crdb/data3/auxiliary/_CRITICAL_ALERT.txt
|
| Checkpoints are created on each node/store hosting this range, to help
| investigate the cause. Only nodes that are more likely to have incorrect data
| are terminated, and usually a majority of replicas continue running. Checkpoints
| are partial, i.e. contain only the data from to the inconsistent range, and
| possibly its neighbouring ranges.
|
| The storage checkpoint directories can/should be deleted when no longer needed.
| They are very helpful in debugging this issue, so before deleting them, please
| consider alternative actions:
|
| - If the store has enough capacity, hold off the deletion until CRDB staff has
| diagnosed the issue.
| - Back up the checkpoints for later investigation.
| - If the stores are nearly full, but the cluster has enough capacity, consider
| gradually decommissioning the affected nodes, to retain the checkpoints.
|
| To inspect the checkpoints, one can use the cockroach debug range-data tool, and
| command line tools like diff. For example:
|
| $ cockroach debug range-data --replicated data/auxiliary/checkpoints/rN_at_M N
|
| Note that a directory that ends with "_pending" might not represent a valid
| checkpoint. Such directories can exist if the node fails during checkpoint
| creation. These directories should be deleted, or inspected with caution.
Error types: (1) *withstack.withStack (2) *errutil.leafError
-- report composition:
*errutil.leafError: log.Fatal: ATTENTION:
replica_consistency.go:832: *withstack.withStack (top exception)
```

Stacktrace (expand for inline code snippets):

[src/runtime/asm_arm64.s#L1221-L1223](src/runtime/asm_arm64.s#L1221-L1223)
[pkg/util/stop/stopper.go#L497-L499](pkg/util/stop/stopper.go#L497-L499)
[pkg/kv/kvserver/replica_consistency.go#L831-L833](pkg/kv/kvserver/replica_consistency.go#L831-L833)

```
src/runtime/asm_arm64.s in runtime.goexit at line 1222
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 498
pkg/kv/kvserver/replica_consistency.go in pkg/kv/kvserver.(*Replica).computeChecksumPostApply.func2 at line 832
```

### Tags

| Tag | Value |
| --- | --- |
| Command | server |
| Environment | v24.3.18 |
| Go Version | go1.22.8 X:nocoverageredesign |
| Platform | linux arm64 |
| Distribution | CCL |
| Cockroach Release | v24.3.18 |
| Cockroach SHA | 37439c410d4583708c898b13a2f9e906f444867b |
| # of CPUs | 48 |
| # of Goroutines | 10442 |

Jira issue: CRDB-59278