ccl/sqlproxyccl: TestCancelQuery failed

ccl/sqlproxyccl.TestCancelQuery [failed](https://mesolite.cluster.engflow.com/invocations/default/97f81ea6-d4dc-4f24-a5c7-902008d14a1f?testReportRun=3&testReportShard=7&testReportAttempt=1#targets-Ly9wa2cvY2NsL3NxbHByb3h5Y2NsOnNxbHByb3h5Y2NsX3Rlc3Q=) on release-25.2.18-rc @ [2f6dda92a21fd70cf96068bab345493ff199f83d](https://github.com/cockroachdb/cockroach/commits/2f6dda92a21fd70cf96068bab345493ff199f83d):

```
pkg/ccl/sqlproxyccl/conn_migration.go:49 +0x31
github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl.(*forwarder).TransferConnection()
pkg/ccl/sqlproxyccl/conn_migration.go:149 +0x124
github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/balancer.(*Balancer).processQueue.func1.1()
pkg/ccl/sqlproxyccl/balancer/balancer.go:371 +0x27b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
pkg/util/stop/stopper.go:507 +0x342
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.gowrap1()
pkg/util/stop/stopper.go:508 +0x4f

Previous write at 0x0000117f5190 by goroutine 4868:
reflect.typedmemmove()
GOROOT/src/runtime/mbarrier.go:225 +0x0
reflect.Value.Set()
GOROOT/src/reflect/value.go:2321 +0x184
github.com/cockroachdb/cockroach/pkg/testutils.TestingHook.func1()
pkg/testutils/hook.go:18 +0x70
runtime.deferreturn()
GOROOT/src/runtime/panic.go:611 +0x5d
testing.tRunner()
GOROOT/src/testing/testing.go:1690 +0x226
testing.(*T).Run.gowrap1()
GOROOT/src/testing/testing.go:1743 +0x44

Goroutine 4871 (running) created at:
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx()
pkg/util/stop/stopper.go:496 +0x72f
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask()
pkg/util/stop/stopper.go:367 +0x1ef
github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/balancer.(*Balancer).processQueue.func1()
pkg/ccl/sqlproxyccl/balancer/balancer.go:363 +0xc0
github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/balancer.(*Balancer).processQueue()
pkg/ccl/sqlproxyccl/balancer/balancer.go:386 +0xa5
github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/balancer.(*Balancer).processQueue-fm()
:1 +0x47
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
pkg/util/stop/stopper.go:507 +0x342
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.gowrap1()
pkg/util/stop/stopper.go:508 +0x4f

Goroutine 4868 (finished) created at:
testing.(*T).Run()
GOROOT/src/testing/testing.go:1743 +0x825
github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl.TestCancelQuery()
pkg/ccl/sqlproxyccl/proxy_handler_test.go:1729 +0x1cde
testing.tRunner()
GOROOT/src/testing/testing.go:1690 +0x226
testing.(*T).Run.gowrap1()
GOROOT/src/testing/testing.go:1743 +0x44
==================
```

Parameters:
- attempt=1
- race=true
- run=3
- shard=7
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

Same failure on other branches

- #167577 ccl/sqlproxyccl: TestCancelQuery failed [C-test-failure O-robot T-cloud-platform branch-release-24.3]
- #166665 ccl/sqlproxyccl: TestCancelQuery failed [C-test-failure O-robot T-cloud-platform branch-release-24.3.30-rc]
- #165202 ccl/sqlproxyccl: TestCancelQuery failed [C-test-failure O-robot T-cloud-platform branch-release-26.1]
- #165007 ccl/sqlproxyccl: TestCancelQuery failed [C-test-failure O-robot T-cloud-platform branch-release-24.3.29-rc]

/cc @cockroachdb/sqlproxy-prs @cockroachdb/server

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestCancelQuery.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-63230

Start with ccl/sqlproxyccl/proxy_handler_test.go around TestCancelQuery at line 1729, then read the stack-traced paths in conn_migration.go and balancer/balancer.go. Reproduce the test with race=true and compare the failing execution with the related failures on other branches. Done means the cancellation test no longer reports this race failure.

Written by the indexing model from the issue text.