cockroachdb / cockroachdb/cockroach

pkg/upgrade/upgrademanager/upgrademanager_test: TestConcurrentMigrationAttempts failed

Open
#164,174 3 comments 0 reactions 0 assignees View on GitHub
branch-master C-test-failure O-robot P-3 T-db-server
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

pkg/upgrade/upgrademanager/upgrademanager_test.TestConcurrentMigrationAttempts [failed](https://mesolite.cluster.engflow.com/invocations/default/1d7d0928-a557-4ab0-a632-c540564111a1?testReportRun=1&testReportShard=1&testReportAttempt=1#targets-Ly9wa2cvdXBncmFkZS91cGdyYWRlbWFuYWdlcjp1cGdyYWRlbWFuYWdlcl90ZXN0) on master @ [03585ea17ad05e1aeadd453320dc7f9036250899](https://github.com/cockroachdb/cockroach/commits/03585ea17ad05e1aeadd453320dc7f9036250899):

```
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).sendPartialBatch(0xc005305808, {0x6cc95b0, 0xc00f42b900}, 0xc00dd80000, {{0xc00a6f2a80, 0xc, 0x24}, {0xc00a6f2a80, 0xd, 0x24}}, ...)
pkg/kv/kvclient/kvcoord/dist_sender.go:2275 +0x5db
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).divideAndSendBatchToRanges(0xc005305808, {0x6cc95b0, 0xc00f42b900}, 0xc00dd80000, {{0xc00a6f2a80, 0xc, 0x24}, {0xc00a6f2a80, 0xd, 0x24}}, ...)
pkg/kv/kvclient/kvcoord/dist_sender.go:1822 +0x35f
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).Send(0xc005305808, {0x6cc86c0, 0xc001480dc0}, 0xc00dd80000)
pkg/kv/kvclient/kvcoord/dist_sender.go:1327 +0x505
github.com/cockroachdb/cockroach/pkg/kv.(*CrossRangeTxnWrapperSender).Send(0xc00394ca08, {0x6cc86c0, 0xc001480dc0}, 0xc00dd80000)
pkg/kv/db.go:224 +0xa2
github.com/cockroachdb/cockroach/pkg/kv.(*DB).sendUsingSender(0xc00394c9a0, {0x6cc86c0, 0xc001480dc0}, 0xc00dd80000, {0x6c7fda0, 0xc00394ca08})
pkg/kv/db.go:1191 +0xe7
github.com/cockroachdb/cockroach/pkg/kv.(*DB).send(...)
pkg/kv/db.go:1174
github.com/cockroachdb/cockroach/pkg/kv.sendAndFill({0x6cc86c0, 0xc001480dc0}, 0xc006af9c10, 0xc00dd5c008)
pkg/kv/db.go:983 +0x107
github.com/cockroachdb/cockroach/pkg/kv.(*DB).Run(0xc00394c9a0, {0x6cc86c0, 0xc001480dc0}, 0xc00dd5c008)
pkg/kv/db.go:1006 +0x65
github.com/cockroachdb/cockroach/pkg/kv/kvserver/txnwait.(*Queue).queryTxnStatus(0xc001de3f00, {0x6cc86c0, 0xc001480dc0}, {{0x4d, 0x74, 0xc4, 0x99, 0x30, 0x80, 0x4b, ...}, ...}, ...)
pkg/kv/kvserver/txnwait/queue.go:1059 +0x1c5
github.com/cockroachdb/cockroach/pkg/kv/kvserver/txnwait.(*Queue).startQueryPusherTxn.func1({0x6cc86c0, 0xc001480dc0})
pkg/kv/kvserver/txnwait/queue.go:985 +0x218
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1({0x6cc86c0, 0xc001480dc0})
pkg/util/stop/stopper.go:440 +0x8a
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 19973
pkg/util/stop/stopper.go:438 +0x125

goroutine 20462 [select, 4 minutes]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/concurrency.watchForNotifications({0x6cc86c0?, 0xc0014805a0?}, 0xc002602a00, 0xc007609650)
pkg/kv/kvserver/concurrency/lock_table_waiter.go:834 +0x6e
created by github.com/cockroachdb/cockroach/pkg/kv/kvserver/concurrency.(*lockTableWaiterImpl).WaitOn.func3 in goroutine 15464
pkg/kv/kvserver/concurrency/lock_table_waiter.go:321 +0x13e

goroutine 20448 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnHeartbeater).heartbeatLoop(0xc00b46f9e8, {0x6cc86c0, 0xc00b739c70})
pkg/kv/kvclient/kvcoord/txn_interceptor_heartbeater.go:410 +0x168
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTask(0xc005a34900, {0x6cc86c0, 0xc00b739c70}, {0x602f3d5, 0x21}, 0xc004f4cfb0)
pkg/util/stop/stopper.go:318 +0x137
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnHeartbeater).startHeartbeatLoopLocked.func1()
pkg/kv/kvclient/kvcoord/txn_interceptor_heartbeater.go:365 +0xd5
created by time.goFunc
GOROOT/src/time/sleep.go:215 +0x2d

goroutine 20423 [select]:
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnHeartbeater).heartbeatLoop(0xc007009de8, {0x6cc86c0, 0xc00bd2a230})
pkg/kv/kvclient/kvcoord/txn_interceptor_heartbeater.go:410 +0x168
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTask(0xc005a34900, {0x6cc86c0, 0xc00bd2a230}, {0x602f3d5, 0x21}, 0xc00e433fb0)
pkg/util/stop/stopper.go:318 +0x137
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*txnHeartbeater).startHeartbeatLoopLocked.func1()
pkg/kv/kvclient/kvcoord/txn_interceptor_heartbeater.go:365 +0xd5
created by time.goFunc
GOROOT/src/time/sleep.go:215 +0x2d
```

Parameters:
- attempt=1
- run=1
- shard=1
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

/cc @cockroachdb/release-eng @cockroachdb/release-eng

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestConcurrentMigrationAttempts.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-60672

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.