Sentry: tenant_cluster.go:328: call to validate with empty instances slice (1) assertion failure Wraps: (2) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/upgrade/upgr...

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: [https://cockroach-labs.sentry.io/issues/6828771683/?referrer=webhooks_plugin](https://cockroach-labs.sentry.io/issues/6828771683/?referrer=webhooks_plugin)

Panic Message:

```
tenant_cluster.go:328: call to validate with empty instances slice
(1) assertion failure
Wraps: (2) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/upgrade/upgradecluster.(*TenantCluster).ValidateAfterUpdateSystemVersion
| pkg/upgrade/upgradecluster/tenant_cluster.go:328
| github.com/cockroachdb/cockroach/pkg/upgrade/upgrademanager.(*Manager).Migrate.func2
| pkg/upgrade/upgrademanager/manager.go:334
| github.com/cockroachdb/cockroach/pkg/sql.setVersionSetting.func1.1
| pkg/sql/set_cluster_setting.go:667
| github.com/cockroachdb/cockroach/pkg/kv.(*Txn).exec
| pkg/kv/txn.go:1132
| github.com/cockroachdb/cockroach/pkg/kv.runTxn
| pkg/kv/db.go:1074
| github.com/cockroachdb/cockroach/pkg/kv.(*DB).TxnWithAdmissionControl
| pkg/kv/db.go:1037
| github.com/cockroachdb/cockroach/pkg/kv.(*DB).Txn
| pkg/kv/db.go:1012
| github.com/cockroachdb/cockroach/pkg/sql.setVersionSetting.func1
| pkg/sql/set_cluster_setting.go:596
| github.com/cockroachdb/cockroach/pkg/upgrade/upgrademanager.(*Manager).Migrate
| pkg/upgrade/upgrademanager/manager.go:501
| github.com/cockroachdb/cockroach/pkg/sql.runMigrationsAndUpgradeVersion
| pkg/sql/set_cluster_setting.go:744
| github.com/cockroachdb/cockroach/pkg/sql.setVersionSetting
| pkg/sql/set_cluster_setting.go:680
| github.com/cockroachdb/cockroach/pkg/sql.writeNonDefaultSettingValue
| pkg/sql/set_cluster_setting.go:518
| github.com/cockroachdb/cockroach/pkg/sql.writeSettingInternal.func1
| pkg/sql/set_cluster_setting.go:451
| github.com/cockroachdb/cockroach/pkg/sql.writeSettingInternal
| pkg/sql/set_cluster_setting.go:470
| github.com/cockroachdb/cockroach/pkg/sql.(*setClusterSettingNode).startExec
| pkg/sql/set_cluster_setting.go:339
| github.com/cockroachdb/cockroach/pkg/sql.startExec
| pkg/sql/plan.go:580
| github.com/cockroachdb/cockroach/pkg/sql.(*planNodeToRowSource).Start
| pkg/sql/plan_node_to_row_source.go:181
| github.com/cockroachdb/cockroach/pkg/sql/colflow.(*FlowCoordinator).Start.func1
| pkg/sql/colflow/flow_coordinator.go:111
| github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
| pkg/sql/colexecerror/error.go:162
| github.com/cockroachdb/cockroach/pkg/sql/colflow.(*FlowCoordinator).Start
| pkg/sql/colflow/flow_coordinator.go:110
| github.com/cockroachdb/cockroach/pkg/sql/execinfra.(*ProcessorBaseNoHelper).Run
| pkg/sql/execinfra/processorsbase.go:744
| github.com/cockroachdb/cockroach/pkg/sql/flowinfra.(*FlowBase).Run
| pkg/sql/flowinfra/flow.go:574
| github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlow).Run
| pkg/sql/colflow/vectorized_flow.go:301
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run
| pkg/sql/distsql_running.go:998
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRun
| pkg/sql/distsql_running.go:2101
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRunAll.func3
| pkg/sql/distsql_running.go:1805
| github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRunAll
| pkg/sql/distsql_running.go:1808
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithDistSQLEngine
| pkg/sql/conn_executor_exec.go:3567
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine
| pkg/sql/conn_executor_exec.go:3081
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState
| pkg/sql/conn_executor_exec.go:1106
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt.func2
| pkg/sql/conn_executor_exec.go:174
| github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithProfiling
| pkg/sql/conn_executor_exec.go:4581
Wraps: (3) call to validate with empty instances slice
Error types: (1) *assert.withAssertionFailure (2) *withstack.withStack (3) *errutil.leafError
-- report composition:
*errutil.leafError: call to validate with empty instances slice
tenant_cluster.go:328: *withstack.withStack (top exception)
*assert.withAssertionFailure
```

Stacktrace (expand for inline code snippets):

[pkg/sql/conn_executor_exec.go#L4580-L4582](pkg/sql/conn_executor_exec.go#L4580-L4582)
[pkg/sql/conn_executor_exec.go#L173-L175](pkg/sql/conn_executor_exec.go#L173-L175)
[pkg/sql/conn_executor_exec.go#L1105-L1107](pkg/sql/conn_executor_exec.go#L1105-L1107)
[pkg/sql/conn_executor_exec.go#L3080-L3082](pkg/sql/conn_executor_exec.go#L3080-L3082)
[pkg/sql/conn_executor_exec.go#L3566-L3568](pkg/sql/conn_executor_exec.go#L3566-L3568)
[pkg/sql/distsql_running.go#L1807-L1809](pkg/sql/distsql_running.go#L1807-L1809)
[pkg/sql/distsql_running.go#L1804-L1806](pkg/sql/distsql_running.go#L1804-L1806)
[pkg/sql/distsql_running.go#L2100-L2102](pkg/sql/distsql_running.go#L2100-L2102)
[pkg/sql/distsql_running.go#L997-L999](pkg/sql/distsql_running.go#L997-L999)
[pkg/sql/colflow/vectorized_flow.go#L300-L302](pkg/sql/colflow/vectorized_flow.go#L300-L302)
[pkg/sql/flowinfra/flow.go#L573-L575](pkg/sql/flowinfra/flow.go#L573-L575)
[pkg/sql/execinfra/processorsbase.go#L743-L745](pkg/sql/execinfra/processorsbase.go#L743-L745)
[pkg/sql/colflow/flow_coordinator.go#L109-L111](pkg/sql/colflow/flow_coordinator.go#L109-L111)
[pkg/sql/colexecerror/error.go#L161-L163](pkg/sql/colexecerror/error.go#L161-L163)
[pkg/sql/colflow/flow_coordinator.go#L110-L112](pkg/sql/colflow/flow_coordinator.go#L110-L112)
[pkg/sql/plan_node_to_row_source.go#L180-L182](pkg/sql/plan_node_to_row_source.go#L180-L182)
[pkg/sql/plan.go#L579-L581](pkg/sql/plan.go#L579-L581)
[pkg/sql/set_cluster_setting.go#L338-L340](pkg/sql/set_cluster_setting.go#L338-L340)
[pkg/sql/set_cluster_setting.go#L469-L471](pkg/sql/set_cluster_setting.go#L469-L471)
[pkg/sql/set_cluster_setting.go#L450-L452](pkg/sql/set_cluster_setting.go#L450-L452)
[pkg/sql/set_cluster_setting.go#L517-L519](pkg/sql/set_cluster_setting.go#L517-L519)
[pkg/sql/set_cluster_setting.go#L679-L681](pkg/sql/set_cluster_setting.go#L679-L681)
[pkg/sql/set_cluster_setting.go#L743-L745](pkg/sql/set_cluster_setting.go#L743-L745)
[pkg/upgrade/upgrademanager/manager.go#L500-L502](pkg/upgrade/upgrademanager/manager.go#L500-L502)
[pkg/sql/set_cluster_setting.go#L595-L597](pkg/sql/set_cluster_setting.go#L595-L597)
[pkg/kv/db.go#L1011-L1013](pkg/kv/db.go#L1011-L1013)
[pkg/kv/db.go#L1036-L1038](pkg/kv/db.go#L1036-L1038)
[pkg/kv/db.go#L1073-L1075](pkg/kv/db.go#L1073-L1075)
[pkg/kv/txn.go#L1131-L1133](pkg/kv/txn.go#L1131-L1133)
[pkg/sql/set_cluster_setting.go#L666-L668](pkg/sql/set_cluster_setting.go#L666-L668)
[pkg/upgrade/upgrademanager/manager.go#L333-L335](pkg/upgrade/upgrademanager/manager.go#L333-L335)
[pkg/upgrade/upgradecluster/tenant_cluster.go#L327-L329](pkg/upgrade/upgradecluster/tenant_cluster.go#L327-L329)

```
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execWithProfiling at line 4581
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmt.func2 at line 174
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmtInOpenState at line 1106
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).dispatchToExecutionEngine at line 3081
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execWithDistSQLEngine at line 3567
pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).PlanAndRunAll at line 1808
pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).PlanAndRunAll.func3 at line 1805
pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).PlanAndRun at line 2101
pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).Run at line 998
pkg/sql/colflow/vectorized_flow.go in pkg/sql/colflow.(*vectorizedFlow).Run at line 301
pkg/sql/flowinfra/flow.go in pkg/sql/flowinfra.(*FlowBase).Run at line 574
pkg/sql/execinfra/processorsbase.go in pkg/sql/execinfra.(*ProcessorBaseNoHelper).Run at line 744
pkg/sql/colflow/flow_coordinator.go in pkg/sql/colflow.(*FlowCoordinator).Start at line 110
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError at line 162
pkg/sql/colflow/flow_coordinator.go in pkg/sql/colflow.(*FlowCoordinator).Start.func1 at line 111
pkg/sql/plan_node_to_row_source.go in pkg/sql.(*planNodeToRowSource).Start at line 181
pkg/sql/plan.go in pkg/sql.startExec at line 580
pkg/sql/set_cluster_setting.go in pkg/sql.(*setClusterSettingNode).startExec at line 339
pkg/sql/set_cluster_setting.go in pkg/sql.writeSettingInternal at line 470
pkg/sql/set_cluster_setting.go in pkg/sql.writeSettingInternal.func1 at line 451
pkg/sql/set_cluster_setting.go in pkg/sql.writeNonDefaultSettingValue at line 518
pkg/sql/set_cluster_setting.go in pkg/sql.setVersionSetting at line 680
pkg/sql/set_cluster_setting.go in pkg/sql.runMigrationsAndUpgradeVersion at line 744
pkg/upgrade/upgrademanager/manager.go in pkg/upgrade/upgrademanager.(*Manager).Migrate at line 501
pkg/sql/set_cluster_setting.go in pkg/sql.setVersionSetting.func1 at line 596
pkg/kv/db.go in pkg/kv.(*DB).Txn at line 1012
pkg/kv/db.go in pkg/kv.(*DB).TxnWithAdmissionControl at line 1037
pkg/kv/db.go in pkg/kv.runTxn at line 1074
pkg/kv/txn.go in pkg/kv.(*Txn).exec at line 1132
pkg/sql/set_cluster_setting.go in pkg/sql.setVersionSetting.func1.1 at line 667
pkg/upgrade/upgrademanager/manager.go in pkg/upgrade/upgrademanager.(*Manager).Migrate.func2 at line 334
pkg/upgrade/upgradecluster/tenant_cluster.go in pkg/upgrade/upgradecluster.(*TenantCluster).ValidateAfterUpdateSystemVersion at line 328
```

### Tags

| Tag | Value |
| --- | --- |
| Command | mt start-sql |
| Environment | v25.3.0 |
| Go Version | go1.23.7 X:nocoverageredesign |
| Platform | linux amd64 |
| Distribution | CCL |
| Cockroach Release | v25.3.0 |
| Cockroach SHA | cb540b101146345c1df9f265f65ea321c62b10b3 |
| # of CPUs | 4 |
| # of Goroutines | 343 |

Jira issue: CRDB-53793