Sentry: datum.go:1216: expected *DString, found tree.dNull (1) attached stack trace -- stack trace: | runtime.gopanic | GOROOT/src/runtime/panic.go:770 | [...repeated from below...] Wraps: (2...

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: [https://cockroach-labs.sentry.io/issues/7447428736/?referrer=webhooks_plugin](https://cockroach-labs.sentry.io/issues/7447428736/?referrer=webhooks_plugin)

Panic Message:

```
datum.go:1216: expected *DString, found tree.dNull
(1) attached stack trace
-- stack trace:
| runtime.gopanic
| GOROOT/src/runtime/panic.go:770
| [...repeated from below...]
Wraps: (2) assertion failure
Wraps: (3) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/sql/sem/tree.MustBeDString
| pkg/sql/sem/tree/datum.go:1216
| github.com/cockroachdb/cockroach/pkg/server.(*adminServer).dataDistributionHelper
| pkg/server/admin.go:3051
| github.com/cockroachdb/cockroach/pkg/server.(*adminServer).DataDistribution
| pkg/server/admin.go:2922
| github.com/cockroachdb/cockroach/pkg/server/serverpb._Admin_DataDistribution_Handler.func1
| bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/github.com/cockroachdb/cockroach/pkg/server/serverpb/admin.pb.go:6242
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.ServerInterceptor.func12
| pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:111
| google.golang.org/grpc.getChainUnaryHandler.func1
| external/org_golang_google_grpc/server.go:1154
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func3
| pkg/rpc/context.go:165
| google.golang.org/grpc.getChainUnaryHandler.func1
| external/org_golang_google_grpc/server.go:1154
| github.com/cockroachdb/cockroach/pkg/rpc.kvAuth.unaryInterceptor
| pkg/rpc/auth.go:90
| google.golang.org/grpc.getChainUnaryHandler.func1
| external/org_golang_google_grpc/server.go:1154
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1.1
| pkg/rpc/context.go:132
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr
| pkg/util/stop/stopper.go:349
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1
| pkg/rpc/context.go:130
| google.golang.org/grpc.NewServer.chainUnaryServerInterceptors.chainUnaryInterceptors.func1
| external/org_golang_google_grpc/server.go:1145
| github.com/cockroachdb/cockroach/pkg/server/serverpb._Admin_DataDistribution_Handler
| bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/github.com/cockroachdb/cockroach/pkg/server/serverpb/admin.pb.go:6244
| google.golang.org/grpc.(*Server).processUnaryRPC
| external/org_golang_google_grpc/server.go:1335
| google.golang.org/grpc.(*Server).handleStream
| external/org_golang_google_grpc/server.go:1712
| google.golang.org/grpc.(*Server).serveStreams.func1.1
| external/org_golang_google_grpc/server.go:947
| runtime.goexit
| src/runtime/asm_amd64.s:1695
Wraps: (4) expected *DString, found tree.dNull
Error types: (1) *withstack.withStack (2) *assert.withAssertionFailure (3) *withstack.withStack (4) *errutil.leafError
-- report composition:
*errutil.leafError: expected *DString, found tree.dNull
datum.go:1216: *withstack.withStack (top exception)
*assert.withAssertionFailure
panic.go:770: *withstack.withStack (1)
(check the extra data payloads)
```

Stacktrace (expand for inline code snippets):

[src/runtime/asm_amd64.s#L1694-L1696](src/runtime/asm_amd64.s#L1694-L1696)
[external/org_golang_google_grpc/server.go#L946-L948](external/org_golang_google_grpc/server.go#L946-L948)
[external/org_golang_google_grpc/server.go#L1711-L1713](external/org_golang_google_grpc/server.go#L1711-L1713)
[external/org_golang_google_grpc/server.go#L1334-L1336](external/org_golang_google_grpc/server.go#L1334-L1336)
[bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/https://github.com/cockroachdb/cockroach/blob/5df37049d6ba68a9eb08b1643a8755f5c4d89d21/pkg/server/serverpb/admin.pb.go#L6243-L6245](bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/https://github.com/cockroachdb/cockroach/blob/5df37049d6ba68a9eb08b1643a8755f5c4d89d21/pkg/server/serverpb/admin.pb.go#L6243-L6245)
[external/org_golang_google_grpc/server.go#L1144-L1146](external/org_golang_google_grpc/server.go#L1144-L1146)
[pkg/rpc/context.go#L129-L131](pkg/rpc/context.go#L129-L131)
[pkg/util/stop/stopper.go#L348-L350](pkg/util/stop/stopper.go#L348-L350)
[pkg/rpc/context.go#L131-L133](pkg/rpc/context.go#L131-L133)
[external/org_golang_google_grpc/server.go#L1153-L1155](external/org_golang_google_grpc/server.go#L1153-L1155)
[pkg/rpc/auth.go#L89-L91](pkg/rpc/auth.go#L89-L91)
[external/org_golang_google_grpc/server.go#L1153-L1155](external/org_golang_google_grpc/server.go#L1153-L1155)
[pkg/rpc/context.go#L164-L166](pkg/rpc/context.go#L164-L166)
[external/org_golang_google_grpc/server.go#L1153-L1155](external/org_golang_google_grpc/server.go#L1153-L1155)
[pkg/util/tracing/grpcinterceptor/grpc_interceptor.go#L110-L112](pkg/util/tracing/grpcinterceptor/grpc_interceptor.go#L110-L112)
[bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/https://github.com/cockroachdb/cockroach/blob/5df37049d6ba68a9eb08b1643a8755f5c4d89d21/pkg/server/serverpb/admin.pb.go#L6241-L6243](bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/https://github.com/cockroachdb/cockroach/blob/5df37049d6ba68a9eb08b1643a8755f5c4d89d21/pkg/server/serverpb/admin.pb.go#L6241-L6243)
[pkg/server/admin.go#L2921-L2923](pkg/server/admin.go#L2921-L2923)
[pkg/server/admin.go#L3050-L3052](pkg/server/admin.go#L3050-L3052)
[pkg/sql/sem/tree/datum.go#L1215-L1217](pkg/sql/sem/tree/datum.go#L1215-L1217)
[GOROOT/src/runtime/panic.go#L769-L771](GOROOT/src/runtime/panic.go#L769-L771)
[src/runtime/asm_amd64.s#L1694-L1696](src/runtime/asm_amd64.s#L1694-L1696)
[external/org_golang_google_grpc/server.go#L946-L948](external/org_golang_google_grpc/server.go#L946-L948)
[external/org_golang_google_grpc/server.go#L1711-L1713](external/org_golang_google_grpc/server.go#L1711-L1713)
[external/org_golang_google_grpc/server.go#L1334-L1336](external/org_golang_google_grpc/server.go#L1334-L1336)
[bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/https://github.com/cockroachdb/cockroach/blob/5df37049d6ba68a9eb08b1643a8755f5c4d89d21/pkg/server/serverpb/admin.pb.go#L6243-L6245](bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/https://github.com/cockroachdb/cockroach/blob/5df37049d6ba68a9eb08b1643a8755f5c4d89d21/pkg/server/serverpb/admin.pb.go#L6243-L6245)
[external/org_golang_google_grpc/server.go#L1144-L1146](external/org_golang_google_grpc/server.go#L1144-L1146)
[pkg/rpc/context.go#L129-L131](pkg/rpc/context.go#L129-L131)
[pkg/util/stop/stopper.go#L348-L350](pkg/util/stop/stopper.go#L348-L350)
[pkg/rpc/context.go#L131-L133](pkg/rpc/context.go#L131-L133)
[external/org_golang_google_grpc/server.go#L1153-L1155](external/org_golang_google_grpc/server.go#L1153-L1155)
[pkg/rpc/auth.go#L89-L91](pkg/rpc/auth.go#L89-L91)
[external/org_golang_google_grpc/server.go#L1153-L1155](external/org_golang_google_grpc/server.go#L1153-L1155)
[pkg/rpc/context.go#L164-L166](pkg/rpc/context.go#L164-L166)
[external/org_golang_google_grpc/server.go#L1153-L1155](external/org_golang_google_grpc/server.go#L1153-L1155)
[pkg/util/tracing/grpcinterceptor/grpc_interceptor.go#L110-L112](pkg/util/tracing/grpcinterceptor/grpc_interceptor.go#L110-L112)
[bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/https://github.com/cockroachdb/cockroach/blob/5df37049d6ba68a9eb08b1643a8755f5c4d89d21/pkg/server/serverpb/admin.pb.go#L6241-L6243](bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/https://github.com/cockroachdb/cockroach/blob/5df37049d6ba68a9eb08b1643a8755f5c4d89d21/pkg/server/serverpb/admin.pb.go#L6241-L6243)
[pkg/server/admin.go#L2921-L2923](pkg/server/admin.go#L2921-L2923)
[pkg/server/admin.go#L3050-L3052](pkg/server/admin.go#L3050-L3052)
[pkg/sql/sem/tree/datum.go#L1215-L1217](pkg/sql/sem/tree/datum.go#L1215-L1217)

```
src/runtime/asm_amd64.s in runtime.goexit at line 1695
external/org_golang_google_grpc/server.go in google.golang.org/grpc.(*Server).serveStreams.func1.1 at line 947
external/org_golang_google_grpc/server.go in google.golang.org/grpc.(*Server).handleStream at line 1712
external/org_golang_google_grpc/server.go in google.golang.org/grpc.(*Server).processUnaryRPC at line 1335
bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/github.com/cockroachdb/cockroach/pkg/server/serverpb/admin.pb.go in pkg/server/serverpb._Admin_DataDistribution_Handler at line 6244
external/org_golang_google_grpc/server.go in google.golang.org/grpc.NewServer.chainUnaryServerInterceptors.chainUnaryInterceptors.func1 at line 1145
pkg/rpc/context.go in pkg/rpc.NewServerEx.func1 at line 130
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunTaskWithErr at line 349
pkg/rpc/context.go in pkg/rpc.NewServerEx.func1.1 at line 132
external/org_golang_google_grpc/server.go in google.golang.org/grpc.getChainUnaryHandler.func1 at line 1154
pkg/rpc/auth.go in pkg/rpc.kvAuth.unaryInterceptor at line 90
external/org_golang_google_grpc/server.go in google.golang.org/grpc.getChainUnaryHandler.func1 at line 1154
pkg/rpc/context.go in pkg/rpc.NewServerEx.func3 at line 165
external/org_golang_google_grpc/server.go in google.golang.org/grpc.getChainUnaryHandler.func1 at line 1154
pkg/util/tracing/grpcinterceptor/grpc_interceptor.go in pkg/rpc.NewServerEx.ServerInterceptor.func12 at line 111
bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/github.com/cockroachdb/cockroach/pkg/server/serverpb/admin.pb.go in pkg/server/serverpb._Admin_DataDistribution_Handler.func1 at line 6242
pkg/server/admin.go in pkg/server.(*adminServer).DataDistribution at line 2922
pkg/server/admin.go in pkg/server.(*adminServer).dataDistributionHelper at line 3051
pkg/sql/sem/tree/datum.go in pkg/sql/sem/tree.MustBeDString at line 1216
GOROOT/src/runtime/panic.go in runtime.gopanic at line 770
src/runtime/asm_amd64.s in runtime.goexit at line 1695
external/org_golang_google_grpc/server.go in google.golang.org/grpc.(*Server).serveStreams.func1.1 at line 947
external/org_golang_google_grpc/server.go in google.golang.org/grpc.(*Server).handleStream at line 1712
external/org_golang_google_grpc/server.go in google.golang.org/grpc.(*Server).processUnaryRPC at line 1335
bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/github.com/cockroachdb/cockroach/pkg/server/serverpb/admin.pb.go in pkg/server/serverpb._Admin_DataDistribution_Handler at line 6244
external/org_golang_google_grpc/server.go in google.golang.org/grpc.NewServer.chainUnaryServerInterceptors.chainUnaryInterceptors.func1 at line 1145
pkg/rpc/context.go in pkg/rpc.NewServerEx.func1 at line 130
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunTaskWithErr at line 349
pkg/rpc/context.go in pkg/rpc.NewServerEx.func1.1 at line 132
external/org_golang_google_grpc/server.go in google.golang.org/grpc.getChainUnaryHandler.func1 at line 1154
pkg/rpc/auth.go in pkg/rpc.kvAuth.unaryInterceptor at line 90
external/org_golang_google_grpc/server.go in google.golang.org/grpc.getChainUnaryHandler.func1 at line 1154
pkg/rpc/context.go in pkg/rpc.NewServerEx.func3 at line 165
external/org_golang_google_grpc/server.go in google.golang.org/grpc.getChainUnaryHandler.func1 at line 1154
pkg/util/tracing/grpcinterceptor/grpc_interceptor.go in pkg/rpc.NewServerEx.ServerInterceptor.func12 at line 111
bazel-out/k8-opt/bin/pkg/server/serverpb/serverpb_go_proto_/github.com/cockroachdb/cockroach/pkg/server/serverpb/admin.pb.go in pkg/server/serverpb._Admin_DataDistribution_Handler.func1 at line 6242
pkg/server/admin.go in pkg/server.(*adminServer).DataDistribution at line 2922
pkg/server/admin.go in pkg/server.(*adminServer).dataDistributionHelper at line 3051
pkg/sql/sem/tree/datum.go in pkg/sql/sem/tree.MustBeDString at line 1216
```

### Tags

| Tag | Value |
| --- | --- |
| Command | server |
| Environment | v24.3.6 |
| Go Version | go1.22.8 X:nocoverageredesign |
| Platform | linux amd64 |
| Distribution | CCL |
| Cockroach Release | v24.3.6 |
| Cockroach SHA | 5df37049d6ba68a9eb08b1643a8755f5c4d89d21 |
| # of CPUs | 92 |
| # of Goroutines | 1661 |

Jira issue: CRDB-63353