cockroachdb / cockroachdb/cockroach
Sentry: replica_command.go:3579: list of voter targets overlaps with the list of non-voter targets: voters: [n1,s1 n4,s4 n7,s7], non-voters: [n4,s4 n9,s9] (1) Wraps: (2) assertion failure Wraps: (3) ...
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.
Sentry Link: [https://cockroach-labs.sentry.io/issues/7309464169/?referrer=webhooks_plugin](https://cockroach-labs.sentry.io/issues/7309464169/?referrer=webhooks_plugin)
Panic Message:
```
replica_command.go:3579: list of voter targets overlaps with the list of non-voter targets: voters: [n1,s1 n4,s4 n7,s7], non-voters: [n4,s4 n9,s9]
(1)
Wraps: (2) assertion failure
Wraps: (3)
| (opaque error wrapper)
| type name: github.com/cockroachdb/errors/withstack/*withstack.withStack
| reportable 0:
|
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).AdminRelocateRange
| pkg/kv/kvserver/replica_command.go:3579
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).executeAdminBatch
| pkg/kv/kvserver/replica_send.go:990
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).SendWithWriteBytes
| pkg/kv/kvserver/replica_send.go:192
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).SendWithWriteBytes
| pkg/kv/kvserver/store_send.go:188
| github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).SendWithWriteBytes
| pkg/kv/kvserver/stores.go:199
| github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal
| node.go:1667
| github.com/cockroachdb/cockroach/pkg/server.(*Node).Batch
| node.go:1860
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func1
| pkg/rpc/context.go:762
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.ServerInterceptor.func12
| pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:60
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
| pkg/rpc/context.go:873
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func3
| pkg/rpc/context.go:182
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
| pkg/rpc/context.go:873
| github.com/cockroachdb/cockroach/pkg/rpc.gatewayRequestRecoveryInterceptor
| pkg/rpc/metrics.go:514
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
| pkg/rpc/context.go:873
| github.com/cockroachdb/cockroach/pkg/server.newGRPCServer.NewRequestMetricsInterceptor.func4
| pkg/rpc/metrics.go:467
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
| pkg/rpc/context.go:873
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1.1
| pkg/rpc/context.go:139
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr
| pkg/util/stop/stopper.go:351
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1
| pkg/rpc/context.go:137
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
| pkg/rpc/context.go:873
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func2
| pkg/rpc/context.go:772
| github.com/cockroachdb/cockroach/pkg/rpc.NewContext.ClientInterceptor.func9
| pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:185
| github.com/cockroachdb/cockroach/pkg/rpc.getChainUnaryInvoker.func1
| pkg/rpc/context.go:957
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func3
| pkg/rpc/context.go:842
| github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.Batch
| pkg/rpc/context.go:965
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*grpcTransport).sendBatch
| pkg/kv/kvclient/kvcoord/transport.go:211
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*grpcTransport).SendNext
| pkg/kv/kvclient/kvcoord/transport.go:189
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).sendToReplicas
| pkg/kv/kvclient/kvcoord/dist_sender.go:2768
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).sendPartialBatch
| pkg/kv/kvclient/kvcoord/dist_sender.go:2227
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).divideAndSendBatchToRanges
| pkg/kv/kvclient/kvcoord/dist_sender.go:1775
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).Send
| pkg/kv/kvclient/kvcoord/dist_sender.go:1280
| github.com/cockroachdb/cockroach/pkg/kv.(*CrossRangeTxnWrapperSender).Send
| pkg/kv/db.go:224
Wraps: (4) list of voter targets overlaps with the list of non-voter targets: voters: [n1,s1 n4,s4 n7,s7], non-voters: [n4,s4 n9,s9]
Error types: (1) *colexecerror.notInternalError (2) *assert.withAssertionFailure (3) *errbase.opaqueWrapper (4) *errutil.leafError
-- report composition:
*errutil.leafError: list of voter targets overlaps with the list of non-voter targets: voters: [n1,s1 n4,s4 n7,s7], non-voters: [n4,s4 n9,s9]
replica_command.go:3579: *withstack.withStack (top exception)
*assert.withAssertionFailure
*colexecerror.notInternalError
```
Stacktrace (expand for inline code snippets):
[pkg/kv/db.go#L223-L225](pkg/kv/db.go#L223-L225)
[pkg/kv/kvclient/kvcoord/dist_sender.go#L1279-L1281](pkg/kv/kvclient/kvcoord/dist_sender.go#L1279-L1281)
[pkg/kv/kvclient/kvcoord/dist_sender.go#L1774-L1776](pkg/kv/kvclient/kvcoord/dist_sender.go#L1774-L1776)
[pkg/kv/kvclient/kvcoord/dist_sender.go#L2226-L2228](pkg/kv/kvclient/kvcoord/dist_sender.go#L2226-L2228)
[pkg/kv/kvclient/kvcoord/dist_sender.go#L2767-L2769](pkg/kv/kvclient/kvcoord/dist_sender.go#L2767-L2769)
[pkg/kv/kvclient/kvcoord/transport.go#L188-L190](pkg/kv/kvclient/kvcoord/transport.go#L188-L190)
[pkg/kv/kvclient/kvcoord/transport.go#L210-L212](pkg/kv/kvclient/kvcoord/transport.go#L210-L212)
[pkg/rpc/context.go#L964-L966](pkg/rpc/context.go#L964-L966)
[pkg/rpc/context.go#L841-L843](pkg/rpc/context.go#L841-L843)
[pkg/rpc/context.go#L956-L958](pkg/rpc/context.go#L956-L958)
[pkg/util/tracing/grpcinterceptor/grpc_interceptor.go#L184-L186](pkg/util/tracing/grpcinterceptor/grpc_interceptor.go#L184-L186)
[pkg/rpc/context.go#L771-L773](pkg/rpc/context.go#L771-L773)
[pkg/rpc/context.go#L872-L874](pkg/rpc/context.go#L872-L874)
[pkg/rpc/context.go#L136-L138](pkg/rpc/context.go#L136-L138)
[pkg/util/stop/stopper.go#L350-L352](pkg/util/stop/stopper.go#L350-L352)
[pkg/rpc/context.go#L138-L140](pkg/rpc/context.go#L138-L140)
[pkg/rpc/context.go#L872-L874](pkg/rpc/context.go#L872-L874)
[pkg/rpc/metrics.go#L466-L468](pkg/rpc/metrics.go#L466-L468)
[pkg/rpc/context.go#L872-L874](pkg/rpc/context.go#L872-L874)
[pkg/rpc/metrics.go#L513-L515](pkg/rpc/metrics.go#L513-L515)
[pkg/rpc/context.go#L872-L874](pkg/rpc/context.go#L872-L874)
[pkg/rpc/context.go#L181-L183](pkg/rpc/context.go#L181-L183)
[pkg/rpc/context.go#L872-L874](pkg/rpc/context.go#L872-L874)
[pkg/util/tracing/grpcinterceptor/grpc_interceptor.go#L59-L61](pkg/util/tracing/grpcinterceptor/grpc_interceptor.go#L59-L61)
[pkg/rpc/context.go#L761-L763](pkg/rpc/context.go#L761-L763)
[node.go#L1859-L1861](node.go#L1859-L1861)
[node.go#L1666-L1668](node.go#L1666-L1668)
[pkg/kv/kvserver/stores.go#L198-L200](pkg/kv/kvserver/stores.go#L198-L200)
[pkg/kv/kvserver/store_send.go#L187-L189](pkg/kv/kvserver/store_send.go#L187-L189)
[pkg/kv/kvserver/replica_send.go#L191-L193](pkg/kv/kvserver/replica_send.go#L191-L193)
[pkg/kv/kvserver/replica_send.go#L989-L991](pkg/kv/kvserver/replica_send.go#L989-L991)
[pkg/kv/kvserver/replica_command.go#L3578-L3580](pkg/kv/kvserver/replica_command.go#L3578-L3580)
```
pkg/kv/db.go in pkg/kv.(*CrossRangeTxnWrapperSender).Send at line 224
pkg/kv/kvclient/kvcoord/dist_sender.go in pkg/kv/kvclient/kvcoord.(*DistSender).Send at line 1280
pkg/kv/kvclient/kvcoord/dist_sender.go in pkg/kv/kvclient/kvcoord.(*DistSender).divideAndSendBatchToRanges at line 1775
pkg/kv/kvclient/kvcoord/dist_sender.go in pkg/kv/kvclient/kvcoord.(*DistSender).sendPartialBatch at line 2227
pkg/kv/kvclient/kvcoord/dist_sender.go in pkg/kv/kvclient/kvcoord.(*DistSender).sendToReplicas at line 2768
pkg/kv/kvclient/kvcoord/transport.go in pkg/kv/kvclient/kvcoord.(*grpcTransport).SendNext at line 189
pkg/kv/kvclient/kvcoord/transport.go in pkg/kv/kvclient/kvcoord.(*grpcTransport).sendBatch at line 211
pkg/rpc/context.go in pkg/rpc.internalClientAdapter.Batch at line 965
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.func3 at line 842
pkg/rpc/context.go in pkg/rpc.getChainUnaryInvoker.func1 at line 957
pkg/util/tracing/grpcinterceptor/grpc_interceptor.go in pkg/rpc.NewContext.ClientInterceptor.func9 at line 185
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.func2 at line 772
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4 at line 873
pkg/rpc/context.go in pkg/rpc.NewServerEx.func1 at line 137
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunTaskWithErr at line 351
pkg/rpc/context.go in pkg/rpc.NewServerEx.func1.1 at line 139
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4 at line 873
pkg/rpc/metrics.go in pkg/server.newGRPCServer.NewRequestMetricsInterceptor.func4 at line 467
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4 at line 873
pkg/rpc/metrics.go in pkg/rpc.gatewayRequestRecoveryInterceptor at line 514
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4 at line 873
pkg/rpc/context.go in pkg/rpc.NewServerEx.func3 at line 182
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4 at line 873
pkg/util/tracing/grpcinterceptor/grpc_interceptor.go in pkg/rpc.NewServerEx.ServerInterceptor.func12 at line 60
pkg/rpc/context.go in pkg/rpc.makeInternalClientAdapter.func1 at line 762
node.go in pkg/server.(*Node).Batch at line 1860
node.go in pkg/server.(*Node).batchInternal at line 1667
pkg/kv/kvserver/stores.go in pkg/kv/kvserver.(*Stores).SendWithWriteBytes at line 199
pkg/kv/kvserver/store_send.go in pkg/kv/kvserver.(*Store).SendWithWriteBytes at line 188
pkg/kv/kvserver/replica_send.go in pkg/kv/kvserver.(*Replica).SendWithWriteBytes at line 192
pkg/kv/kvserver/replica_send.go in pkg/kv/kvserver.(*Replica).executeAdminBatch at line 990
pkg/kv/kvserver/replica_command.go in pkg/kv/kvserver.(*Replica).AdminRelocateRange at line 3579
```
### Tags
| Tag | Value |
| --- | --- |
| Command | demo |
| Environment | v25.4.1 |
| Go Version | go1.23.12 X:nocoverageredesign |
| Platform | darwin arm64 |
| Distribution | CCL |
| Cockroach Release | v25.4.1 |
| Cockroach SHA | 60d802bd0bcb2c1aa81a4b5fd73fb4aa13bb5114 |
| # of CPUs | 16 |
| # of Goroutines | 9522 |
Jira issue: CRDB-61041
Contributor guide
Assessment
This issue has not been assessed yet.