release-26.3: pkg/sql/logictest/tests/5node-disk/5node-disk_test: TestLogic_hash_join_dist failed

pkg/sql/logictest/tests/5node-disk/5node-disk_test.TestLogic_hash_join_dist [failed](https://mesolite.cluster.engflow.com/invocations/default/b5d984ee-78d8-4e54-9545-4ba9895c90d5?testReportRun=3&testReportShard=14&testReportAttempt=1#targets-Ly9wa2cvc3FsL2xvZ2ljdGVzdC90ZXN0cy81bm9kZS1kaXNrOjVub2RlLWRpc2tfdGVzdA==) on release-26.3 @ [f23f7d6b191aed1ead3768fbfd94d04df78587f0](https://github.com/cockroachdb/cockroach/commits/f23f7d6b191aed1ead3768fbfd94d04df78587f0):

```
=== RUN TestLogic_hash_join_dist
test_log_scope.go:172: test logs captured to: outputs.zip/logTestLogic_hash_join_dist2883780389
test_log_scope.go:82: use -show-logs to present logs inline
test_server_shim.go:146: using test tenant configuration from testserver factory defaults
test_server_shim.go:150: cluster virtualization disabled due to issue: #156124 (expected label: C-bug)
[10:13:59] setting distsql_workmem='36910B';
[10:14:06] --- progress: /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/bin/pkg/sql/logictest/tests/5node-disk/5node-disk_test_/5node-disk_test.runfiles/com_github_cockroachdb_cockroach/pkg/sql/logictest/testdata/logic_test/hash_join_dist: 3 statements
[10:14:17] --- progress: /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/bin/pkg/sql/logictest/tests/5node-disk/5node-disk_test_/5node-disk_test.runfiles/com_github_cockroachdb_cockroach/pkg/sql/logictest/testdata/logic_test/hash_join_dist: 7 statements
[10:14:21] --- progress: /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/bin/pkg/sql/logictest/tests/5node-disk/5node-disk_test_/5node-disk_test.runfiles/com_github_cockroachdb_cockroach/pkg/sql/logictest/testdata/logic_test/hash_join_dist: 13 statements
[10:15:06] --- done: /mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/bin/pkg/sql/logictest/tests/5node-disk/5node-disk_test_/5node-disk_test.runfiles/com_github_cockroachdb_cockroach/pkg/sql/logictest/testdata/logic_test/hash_join_dist with config 5node-disk: 13 tests, 0 failures
logic.go:4863:
/mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/bin/pkg/sql/logictest/tests/5node-disk/5node-disk_test_/5node-disk_test.runfiles/com_github_cockroachdb_cockroach/pkg/sql/logictest/testdata/logic_test/hash_join_dist:53: error while processing
logic.go:4863:
/mnt/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/bin/pkg/sql/logictest/tests/5node-disk/5node-disk_test_/5node-disk_test.runfiles/com_github_cockroachdb_cockroach/pkg/sql/logictest/testdata/logic_test/hash_join_dist:53:
expected success, but found
(XXUUU) none of the remaining voters [n1,s1] are legal additions to (n3,s3):1
panic.go:694: -- test log scope end --
test logs left over in: outputs.zip/logTestLogic_hash_join_dist2883780389
--- FAIL: TestLogic_hash_join_dist (151.81s)
```

Parameters:
- attempt=1
- race=true
- run=3
- shard=14
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestLogic_hash_join_dist.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-68037

Start with pkg/sql/logictest/tests/5node-disk/5node-disk_test and the TestLogic_hash_join_dist failure, then inspect testdata/logic_test/hash_join_dist at line 53 and the captured outputs.zip logs. Reproduce the race-enabled test and investigate the “none of the remaining voters” error. Done means the test completes successfully without this failure.

Written by the indexing model from the issue text.