cockroachdb / cockroachdb/cockroach

Sentry: replica_eval_context.go:106: log.Fatal: can't create a ReplicaEvalContext with assertions but no SpanSet (1) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/kv/...

Open
#169,465 1 comment 0 reactions 0 assignees View on GitHub
branch-release-25.3 C-bug O-sentry T-kv X-blathers-triaged
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: [https://cockroach-labs.sentry.io/issues/7452635978/?referrer=webhooks_plugin](https://cockroach-labs.sentry.io/issues/7452635978/?referrer=webhooks_plugin)

Panic Message:

```
replica_eval_context.go:106: log.Fatal: can't create a ReplicaEvalContext with assertions but no SpanSet
(1) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.NewReplicaEvalContext
| pkg/kv/kvserver/replica_eval_context.go:106
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).evaluate1PC
| pkg/kv/kvserver/replica_write.go:550
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).evaluateWriteBatch
| pkg/kv/kvserver/replica_write.go:443
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).evaluateProposal
| pkg/kv/kvserver/replica_proposal.go:999
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).requestToProposal
| pkg/kv/kvserver/replica_proposal.go:1095
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).evalAndPropose
| pkg/kv/kvserver/replica_raft.go:117
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).executeWriteBatch
| pkg/kv/kvserver/replica_write.go:188
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).executeBatchWithConcurrencyRetries
| pkg/kv/kvserver/replica_send.go:510
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).SendWithWriteBytes
| pkg/kv/kvserver/replica_send.go:189
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).SendWithWriteBytes
| pkg/kv/kvserver/store_send.go:188
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).SendWithWriteBytes
| pkg/kv/kvserver/stores.go:199
| github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal
| pkg/server/node.go:1665
| github.com/cockroachdb/cockroach/pkg/server.(*Node).Batch
| pkg/server/node.go:1858
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func1
| pkg/rpc/context.go:735
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.ServerInterceptor.func12
| pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:96
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
| pkg/rpc/context.go:846
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func3
| pkg/rpc/context.go:178
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
| pkg/rpc/context.go:846
| github.com/cockroachdb/cockroach/pkg/rpc.gatewayRequestRecoveryInterceptor
| pkg/rpc/metrics.go:462
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
| pkg/rpc/context.go:846
| github.com/cockroachdb/cockroach/pkg/server.newGRPCServer.NewRequestMetricsInterceptor.func4
| pkg/rpc/metrics.go:415
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
| pkg/rpc/context.go:846
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1.1
| pkg/rpc/context.go:136
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr
| pkg/util/stop/stopper.go:349
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1
| pkg/rpc/context.go:134
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
| pkg/rpc/context.go:846
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func2
| pkg/rpc/context.go:745
| github.com/cockroachdb/cockroach/pkg/rpc.NewContext.ClientInterceptor.func8
| pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:236
| github.com/cockroachdb/cockroach/pkg/rpc.getChainUnaryInvoker.func1
| pkg/rpc/context.go:930
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func3
| pkg/rpc/context.go:815
| github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.Batch
| pkg/rpc/context.go:938
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*grpcTransport).sendBatch
| pkg/kv/kvclient/kvcoord/transport.go:211
Wraps: (2) log.Fatal: can't create a ReplicaEvalContext with assertions but no SpanSet
Error types: (1) *withstack.withStack (2) *errutil.leafError
-- report composition:
*errutil.leafError: log.Fatal: can't create a ReplicaEvalContext with assertions but no SpanSet
replica_eval_context.go:106: *withstack.withStack (top exception)
```

Stacktrace (expand for inline code snippets):

[pkg/kv/kvclient/kvcoord/transport.go#L210-L212](pkg/kv/kvclient/kvcoord/transport.go#L210-L212)
[pkg/rpc/context.go#L937-L939](pkg/rpc/context.go#L937-L939)
[pkg/rpc/context.go#L814-L816](pkg/rpc/context.go#L814-L816)
[pkg/rpc/context.go#L929-L931](pkg/rpc/context.go#L929-L931)
[pkg/util/tracing/grpcinterceptor/grpc_interceptor.go#L235-L237](pkg/util/tracing/grpcinterceptor/grpc_interceptor.go#L235-L237)
[pkg/rpc/context.go#L744-L746](pkg/rpc/context.go#L744-L746)
[pkg/rpc/context.go#L845-L847](pkg/rpc/context.go#L845-L847)
[pkg/rpc/context.go#L133-L135](pkg/rpc/context.go#L133-L135)
[pkg/util/stop/stopper.go#L348-L350](pkg/util/stop/stopper.go#L348-L350)
[pkg/rpc/context.go#L135-L137](pkg/rpc/context.go#L135-L137)
[pkg/rpc/context.go#L845-L847](pkg/rpc/context.go#L845-L847)
[pkg/rpc/metrics.go#L414-L416](pkg/rpc/metrics.go#L414-L416)
[pkg/rpc/context.go#L845-L847](pkg/rpc/context.go#L845-L847)
[pkg/rpc/metrics.go#L461-L463](pkg/rpc/metrics.go#L461-L463)
[pkg/rpc/context.go#L845-L847](pkg/rpc/context.go#L845-L847)
[pkg/rpc/context.go#L177-L179](pkg/rpc/context.go#L177-L179)
[pkg/rpc/context.go#L845-L847](pkg/rpc/context.go#L845-L847)
[pkg/util/tracing/grpcinterceptor/grpc_interceptor.go#L95-L97](pkg/util/tracing/grpcinterceptor/grpc_interceptor.go#L95-L97)
[pkg/rpc/context.go#L734-L736](pkg/rpc/context.go#L734-L736)
[pkg/server/node.go#L1857-L1859](pkg/server/node.go#L1857-L1859)
[pkg/server/node.go#L1664-L1666](pkg/server/node.go#L1664-L1666)
[pkg/kv/kvserver/stores.go#L198-L200](pkg/kv/kvserver/stores.go#L198-L200)
[pkg/kv/kvserver/store_send.go#L187-L189](pkg/kv/kvserver/store_send.go#L187-L189)
[pkg/kv/kvserver/replica_send.go#L188-L190](pkg/kv/kvserver/replica_send.go#L188-L190)
[pkg/kv/kvserver/replica_send.go#L509-L511](pkg/kv/kvserver/replica_send.go#L509-L511)
[pkg/kv/kvserver/replica_write.go#L187-L189](pkg/kv/kvserver/replica_write.go#L187-L189)
[pkg/kv/kvserver/replica_raft.go#L116-L118](pkg/kv/kvserver/replica_raft.go#L116-L118)
[pkg/kv/kvserver/replica_proposal.go#L1094-L1096](pkg/kv/kvserver/replica_proposal.go#L1094-L1096)
[pkg/kv/kvserver/replica_proposal.go#L998-L1000](pkg/kv/kvserver/replica_proposal.go#L998-L1000)
[pkg/kv/kvserver/replica_write.go#L442-L444](pkg/kv/kvserver/replica_write.go#L442-L444)
[pkg/kv/kvserver/replica_write.go#L549-L551](pkg/kv/kvserver/replica_write.go#L549-L551)
[pkg/kv/kvserver/replica_eval_context.go#L105-L107](pkg/kv/kvserver/replica_eval_context.go#L105-L107)

```
pkg/kv/kvclient/kvcoord/transport.go in pkg/kv/kvclient/kvcoord.(*grpcTransport).sendBatch at line 211
pkg/rpc/context.go in pkg/rpc.internalClientAdapter.Batch at line 938
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.func3 at line 815
pkg/rpc/context.go in pkg/rpc.getChainUnaryInvoker.func1 at line 930
pkg/util/tracing/grpcinterceptor/grpc_interceptor.go in pkg/rpc.NewContext.ClientInterceptor.func8 at line 236
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.func2 at line 745
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4 at line 846
pkg/rpc/context.go in pkg/rpc.NewServerEx.func1 at line 134
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunTaskWithErr at line 349
pkg/rpc/context.go in pkg/rpc.NewServerEx.func1.1 at line 136
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4 at line 846
pkg/rpc/metrics.go in pkg/server.newGRPCServer.NewRequestMetricsInterceptor.func4 at line 415
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4 at line 846
pkg/rpc/metrics.go in pkg/rpc.gatewayRequestRecoveryInterceptor at line 462
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4 at line 846
pkg/rpc/context.go in pkg/rpc.NewServerEx.func3 at line 178
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4 at line 846
pkg/util/tracing/grpcinterceptor/grpc_interceptor.go in pkg/rpc.NewServerEx.ServerInterceptor.func12 at line 96
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.func1 at line 735
pkg/server/node.go in pkg/server.(*Node).Batch at line 1858
pkg/server/node.go in pkg/server.(*Node).batchInternal at line 1665
pkg/kv/kvserver/stores.go in pkg/kv/kvserver.(*Stores).SendWithWriteBytes at line 199
pkg/kv/kvserver/store_send.go in pkg/kv/kvserver.(*Store).SendWithWriteBytes at line 188
pkg/kv/kvserver/replica_send.go in pkg/kv/kvserver.(*Replica).SendWithWriteBytes at line 189
pkg/kv/kvserver/replica_send.go in pkg/kv/kvserver.(*Replica).executeBatchWithConcurrencyRetries at line 510
pkg/kv/kvserver/replica_write.go in pkg/kv/kvserver.(*Replica).executeWriteBatch at line 188
pkg/kv/kvserver/replica_raft.go in pkg/kv/kvserver.(*Replica).evalAndPropose at line 117
pkg/kv/kvserver/replica_proposal.go in pkg/kv/kvserver.(*Replica).requestToProposal at line 1095
pkg/kv/kvserver/replica_proposal.go in pkg/kv/kvserver.(*Replica).evaluateProposal at line 999
pkg/kv/kvserver/replica_write.go in pkg/kv/kvserver.(*Replica).evaluateWriteBatch at line 443
pkg/kv/kvserver/replica_write.go in pkg/kv/kvserver.(*Replica).evaluate1PC at line 550
pkg/kv/kvserver/replica_eval_context.go in pkg/kv/kvserver.NewReplicaEvalContext at line 106
```

### Tags

| Tag | Value |
| --- | --- |
| Command | start-single-node |
| Environment | v25.3.6 |
| Go Version | go1.23.12 X:nocoverageredesign |
| Platform | linux amd64 |
| Distribution | CCL |
| Cockroach Release | v25.3.6 |
| Cockroach SHA | 7bad8e82ef2989b3cd657baca02567a892c1b5c3 |
| # of CPUs | 384 |
| # of Goroutines | 3168 |

Jira issue: CRDB-63464

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.