cockroachdb / cockroachdb/cockroach
basalt-26.3: kv/kvnemesis: TestKVNemesisMultiNode_Partition_Liveness failed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
kv/kvnemesis.TestKVNemesisMultiNode_Partition_Liveness [failed](https://mesolite.cluster.engflow.com/invocations/default/d0edd2ec-f550-4935-af39-b83ec698ec6d?testReportRun=8&testReportShard=1&testReportAttempt=1#targets-Ly9wa2cva3Yva3ZuZW1lc2lzOmt2bmVtZXNpc190ZXN0) on basalt-26.3 @ [017b84cd0101b832a6ee0c1934d073d108d0b24a](https://github.com/cockroachdb/cockroach/commits/017b84cd0101b832a6ee0c1934d073d108d0b24a):
```
github.com/cockroachdb/cockroach/pkg/kv.(*Txn).exec(0x18e2fcad6c30, {0x6b12760, 0x18e304277d60}, 0x18e2ff1ab218)
pkg/kv/txn.go:1190 +0x9e
github.com/cockroachdb/cockroach/pkg/kv.runTxn({0x6b12760, 0x18e304277d60}, 0x18e2fcad6c30, 0x1?)
pkg/kv/db.go:1169 +0x3e
github.com/cockroachdb/cockroach/pkg/kv.(*DB).TxnWithAdmissionControl(0x18e3055a0280?, {0x6b12760, 0x18e304277d60}, 0xe3df760?, 0xe3?, 0x0, 0x18e2ff1ab218)
pkg/kv/db.go:1129 +0xfd
github.com/cockroachdb/cockroach/pkg/kv.(*DB).Txn(...)
pkg/kv/db.go:1101
github.com/cockroachdb/cockroach/pkg/kv.(*CrossRangeTxnWrapperSender).Send(0x18e2fd467038, {0x6b12760, 0x18e304277d60}, 0x18e30e3df760)
pkg/kv/db.go:246 +0x1a6
github.com/cockroachdb/cockroach/pkg/kv.(*DB).sendUsingSender(0x18e2fd466fd0, {0x6b12760, 0x18e304277d60}, 0x18e30e3df760, {0x6ac1240, 0x18e2fd467038})
pkg/kv/db.go:1250 +0xe7
github.com/cockroachdb/cockroach/pkg/kv.(*DB).send(0x18e2fd466fd0?, {0x6b12760?, 0x18e304277d60?}, 0x18e30e3df760?)
pkg/kv/db.go:1233 +0xaf
github.com/cockroachdb/cockroach/pkg/kv.sendAndFill({0x6b12760, 0x18e304277d60}, 0x18e2ff1ab388, 0x18e2ff81cc08)
pkg/kv/db.go:1027 +0x138
github.com/cockroachdb/cockroach/pkg/kv.(*DB).Run(0x18e2fd466fd0, {0x6b12760, 0x18e304277d60}, 0x18e2ff81cc08)
pkg/kv/db.go:1050 +0x65
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.batchRun(...)
pkg/kv/kvnemesis/applier.go:369
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.dbRunWithResultAndTimestamp({0x6b12760, 0x18e304277d60}, {0x7f7516c42a98, 0x18e2fd466fd0}, {0x18e2ff1ab7e8, 0x1, 0x18e2ff1ab488?})
pkg/kv/kvnemesis/applier.go:359 +0xa5
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.applyClientOp({0x6b12760, 0x18e304277d60}, {0x6b883c0, 0x18e2fd466fd0}, 0x0?, 0x0, 0x0)
pkg/kv/kvnemesis/applier.go:526 +0xc37
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*Applier).applyOp(0x18e2e6c711c0, {0x6b12760, 0x18e304277d60}, 0x18e2fd466fd0, 0x18e304dba140)
pkg/kv/kvnemesis/applier.go:151 +0x180c
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*Applier).Apply(0x18e2e6c711c0, {0x6b12258, 0x18e2f9fbd1d0}, 0x18e304dba140)
pkg/kv/kvnemesis/applier.go:68 +0x134
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.RunNemesis.func2.1(...)
pkg/kv/kvnemesis/kvnemesis.go:164
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.RunNemesis.func2({0x6b12258, 0x18e2f9fbd1d0}, 0x4)
pkg/kv/kvnemesis/kvnemesis.go:188 +0x654
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.GroupWorkers.func1({0x6b12258?, 0x18e2f9fbd1d0?})
pkg/util/ctxgroup/ctxgroup.go:209 +0x25
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.GroupWorkers.Group.GoCtx.func2()
pkg/util/ctxgroup/ctxgroup.go:200 +0x79
golang.org/x/sync/errgroup.(*Group).Go.func1()
external/org_golang_x_sync/errgroup/errgroup.go:93 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 99568
external/org_golang_x_sync/errgroup/errgroup.go:78 +0x95
goroutine 961624 [select, "gen":"4", "mux_n":"2", "n":"2", "rangefeed":"tenant-settings-watcher"]:
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*BufferedSender).run(0x18e31037a780, {0x6b12760, 0x18e31d370d20}, 0x18e3050afcb0, 0x18e33be1a290)
pkg/kv/kvserver/rangefeed/buffered_sender.go:276 +0x15a
github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*StreamManager).Start.func1({0x6b12760, 0x18e31d370d20})
pkg/kv/kvserver/rangefeed/stream_manager.go:196 +0xc3
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func1({0x6b12760, 0x18e31d370d20})
pkg/util/stop/stopper.go:447 +0x8a
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 961621
pkg/util/stop/stopper.go:445 +0x125
```
Parameters:
- attempt=1
- run=8
- shard=1
Help
See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)
Same failure on other branches
- #174430 kv/kvnemesis: TestKVNemesisMultiNode_Partition_Liveness failed [DistSender livelock on partition] [A-kv C-bug C-test-failure O-robot P-3 T-kv branch-master]
/cc @cockroachlabs/kv-triage
[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestKVNemesisMultiNode_Partition_Liveness.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Jira issue: CRDB-68026
Contributor guide
Assessment
This issue has not been assessed yet.