kv/kvnemesis: TestKVNemesisMultiNode_Restart_Liveness failed

kv/kvnemesis.TestKVNemesisMultiNode_Restart_Liveness [failed](https://mesolite.cluster.engflow.com/invocations/default/1c9c04b4-0874-485e-9047-323fc080fc64?testReportRun=7&testReportShard=1&testReportAttempt=1#targets-Ly9wa2cva3Yva3ZuZW1lc2lzOmt2bmVtZXNpc190ZXN0) on master @ [b569625b4f7e1f7850a46dad528a9e9daaba301d](https://github.com/cockroachdb/cockroach/commits/b569625b4f7e1f7850a46dad528a9e9daaba301d):

```
github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*TxnCoordSender).Send(0x6bccd6a2a08, {0x6c52640, 0x6bcc214cb40}, 0x6bce182c000)
pkg/kv/kvclient/kvcoord/txn_coord_sender.go:574 +0x5ea
github.com/cockroachdb/cockroach/pkg/kv.(*DB).sendUsingSender(0x6bcb4b9ce70, {0x6c52640, 0x6bcc214cb40}, 0x6bce182c000, {0x7f73a4ea1a30, 0x6bccd6a2a08})
pkg/kv/db.go:1250 +0xe7
github.com/cockroachdb/cockroach/pkg/kv.(*Txn).Send(0x6bcd04d1450, {0x6c52640, 0x6bcc214cb40}, 0x6bce182c000)
pkg/kv/txn.go:1447 +0x2cc
github.com/cockroachdb/cockroach/pkg/kv.sendAndFill({0x6c52640, 0x6bcc214cb40}, 0x6bcb8a70c40, 0x6bcb5c36608)
pkg/kv/db.go:1027 +0x138
github.com/cockroachdb/cockroach/pkg/kv.(*Txn).Run(0x6bcd04d1450, {0x6c52640, 0x6bcc214cb40}, 0x6bcb5c36608)
pkg/kv/txn.go:920 +0x65
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.batchRun(...)
pkg/kv/kvnemesis/applier.go:392
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.dbRunWithResultAndTimestamp({0x6c52640, 0x6bcc214cb40}, {0x7f73a4102ee0, 0x6bcd04d1450}, {0x6bcb8a710a0, 0x1, 0x0?})
pkg/kv/kvnemesis/applier.go:382 +0xa5
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.applyClientOp.func1(...)
pkg/kv/kvnemesis/applier.go:418
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.applyClientOp({0x6c52640, 0x6bcc214cb40}, {0x6cc9280, 0x6bcd04d1450}, 0x6bcc6a8a840?, 0x1, 0x6bcb8a712c0, 0x0)
pkg/kv/kvnemesis/applier.go:492 +0x2ac6
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*Applier).applyOp.func1({0x6c52640, 0x6bcc214cb40}, 0x6bcd04d1450)
pkg/kv/kvnemesis/applier.go:279 +0x665
github.com/cockroachdb/cockroach/pkg/kv.(*Txn).exec(0x6bcd04d1450, {0x6c52640, 0x6bcc214cb40}, 0x6bcb8a71ae0)
pkg/kv/txn.go:1190 +0x9e
github.com/cockroachdb/cockroach/pkg/kv.runTxn({0x6c52640, 0x6bcc214cb40}, 0x6bcd04d1450, 0x1?)
pkg/kv/db.go:1169 +0x3e
github.com/cockroachdb/cockroach/pkg/kv.(*DB).TxnWithAdmissionControl(0xf4240?, {0x6c52640, 0x6bcc214cb40}, 0x33333333?, 0x33?, 0x0, 0x6bcb8a71ae0)
pkg/kv/db.go:1129 +0xfd
github.com/cockroachdb/cockroach/pkg/kv.(*DB).Txn(...)
pkg/kv/db.go:1101
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*Applier).applyOp(0x6bce4b45540, {0x6c52640, 0x6bcc214cb40}, 0x6bcb4b9ce70, 0x6bcb3bf2d80)
pkg/kv/kvnemesis/applier.go:221 +0x114c
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*Applier).Apply(0x6bce4b45540, {0x6c52138, 0x6bcfd5a6aa0}, 0x6bcb3bf2d80)
pkg/kv/kvnemesis/applier.go:68 +0x134
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.RunNemesis.func2.1(...)
pkg/kv/kvnemesis/kvnemesis.go:164
github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.RunNemesis.func2({0x6c52138, 0x6bcfd5a6aa0}, 0x0)
pkg/kv/kvnemesis/kvnemesis.go:188 +0x653
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.GroupWorkers.func1({0x6c52138?, 0x6bcfd5a6aa0?})
pkg/util/ctxgroup/ctxgroup.go:209 +0x25
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.GroupWorkers.Group.GoCtx.func2()
pkg/util/ctxgroup/ctxgroup.go:200 +0x79
golang.org/x/sync/errgroup.(*Group).Go.func1()
external/org_golang_x_sync/errgroup/errgroup.go:93 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 137211
external/org_golang_x_sync/errgroup/errgroup.go:78 +0x95

goroutine 158494 [select, 10 minutes, "remote_node_id":"2", "workload.id":"RAFT"]:
google.golang.org/grpc.newClientStreamWithParams.func4()
external/org_golang_google_grpc/stream.go:383 +0x8e
created by google.golang.org/grpc.newClientStreamWithParams in goroutine 158493
external/org_golang_google_grpc/stream.go:382 +0xe0b
```

Parameters:
- attempt=1
- run=7
- shard=1
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

/cc @cockroachlabs/kv-triage

[Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-67188