basalt-26.3: roachtest: ycsb/E/nodes=3/binary=basalt/storage=basalt/basalt_nodes=3/rsengine=enabled failed

roachtest.ycsb/E/nodes=3/binary=basalt/storage=basalt/basalt_nodes=3/rsengine=enabled [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyBenchmarksAwsBasaltBazel/21516296?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyBenchmarksAwsBasaltBazel/21516296?buildTab=artifacts#/ycsb/E/nodes=3/binary=basalt/storage=basalt/basalt_nodes=3/rsengine=enabled) on basalt-26.3 @ [2327de0124eca58e8ea82a577a805766789a48e5](https://github.com/cockroachdb/cockroach/commits/2327de0124eca58e8ea82a577a805766789a48e5):

Failed with:

```
(cluster.go:4230).Start: COMMAND_PROBLEM: exit status 1
test artifacts and logs in: /artifacts/ycsb/E/nodes=3/binary=basalt/storage=basalt/basalt_nodes=3/rsengine=enabled/run_1
```
Fatal entries found in Cockroach logs:

```
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 MANIFEST flush failed: version 0: unsupported protocol version
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !goroutine 97 [running]:
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/allstacks.GetWithBuf({0x0?, 0x2106086b7798?, 0xd6afcb?})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/allstacks/allstacks.go:38 +0x125
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/allstacks.Get(...)
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/allstacks/allstacks.go:19
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/log.(*loggerT).outputLogEntry(_, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}}, ...})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/log/clog.go:294 +0xc6
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x86e51c8, 0x2106090c4640}, 0x3, 0x4, 0x3, 0x0?, {0x8630d28, 0x2}, {0x210605b70180, 0x1, ...})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/log/channels.go:104 +0x650
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 MANIFEST flush failed: version 0: unsupported protocol version
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !goroutine 97 [running]:
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/allstacks.GetWithBuf({0x0?, 0x2106086b7798?, 0xd6afcb?})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/allstacks/allstacks.go:38 +0x125
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/allstacks.Get(...)
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/allstacks/allstacks.go:19
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/log.(*loggerT).outputLogEntry(_, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}}, ...})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/log/clog.go:294 +0xc6
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x86e51c8, 0x2106090c4640}, 0x3, 0x4, 0x3, 0x0?, {0x8630d28, 0x2}, {0x210605b70180, 0x1, ...})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/log/channels.go:104 +0x650
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 MANIFEST flush failed: version 0: unsupported protocol version
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !goroutine 97 [running]:
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/allstacks.GetWithBuf({0x0?, 0x2106086b7798?, 0xd6afcb?})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/allstacks/allstacks.go:38 +0x125
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/allstacks.Get(...)
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/allstacks/allstacks.go:19
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/log.(*loggerT).outputLogEntry(_, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}}, ...})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/log/clog.go:294 +0xc6
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x86e51c8, 0x2106090c4640}, 0x3, 0x4, 0x3, 0x0?, {0x8630d28, 0x2}, {0x210605b70180, 0x1, ...})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/log/channels.go:104 +0x650
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 [n?,s?,pebble] 27 MANIFEST flush failed: version 0: unsupported protocol version
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 [n?,s?,pebble] 27 !goroutine 97 [running]:
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 [n?,s?,pebble] 27 !github.com/cockroachdb/cockroach/pkg/util/allstacks.GetWithBuf({0x0?, 0x2106086b7798?, 0xd6afcb?})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 [n?,s?,pebble] 27 ! pkg/util/allstacks/allstacks.go:38 +0x125
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 [n?,s?,pebble] 27 !github.com/cockroachdb/cockroach/pkg/util/allstacks.Get(...)
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 [n?,s?,pebble] 27 ! pkg/util/allstacks/allstacks.go:19
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 [n?,s?,pebble] 27 !github.com/cockroachdb/cockroach/pkg/util/log.(*loggerT).outputLogEntry(_, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}}, ...})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 [n?,s?,pebble] 27 ! pkg/util/log/clog.go:294 +0xc6
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 [n?,s?,pebble] 27 !github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x86e51c8, 0x2106090c4640}, 0x3, 0x4, 0x3, 0x0?, {0x8630d28, 0x2}, {0x210605b70180, 0x1, ...})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 [n?,s?,pebble] 27 ! pkg/util/log/channels.go:104 +0x650
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 MANIFEST flush failed: version 0: unsupported protocol version
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !goroutine 97 [running]:
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/allstacks.GetWithBuf({0x0?, 0x2106086b7798?, 0xd6afcb?})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/allstacks/allstacks.go:38 +0x125
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/allstacks.Get(...)
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/allstacks/allstacks.go:19
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/log.(*loggerT).outputLogEntry(_, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}}, ...})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/log/clog.go:294 +0xc6
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x86e51c8, 0x2106090c4640}, 0x3, 0x4, 0x3, 0x0?, {0x8630d28, 0x2}, {0x210605b70180, 0x1, ...})
F260914 23:20:09.707343 97 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/log/channels.go:104 +0x650
```
Cluster Node to Ip Mapping:

| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21516296-1789423542-22-n4cpu8-0001 | 18.191.176.56 | 10.12.11.170 |
| teamcity-21516296-1789423542-22-n4cpu8-0002 | 18.224.55.246 | 10.12.15.21 |
| teamcity-21516296-1789423542-22-n4cpu8-0003 | 13.59.167.103 | 10.12.1.151 |
| teamcity-21516296-1789423542-22-n4cpu8-0004 | 18.118.4.228 | 10.12.9.200 |

Parameters:
- MetamorphicBasaltMultiAZ=false
- arch=amd64
- cloud=aws
- coverageBuild=false
- cpu=8
- diskCount=0
- encrypted=false
- fs=ext4
- localSSD=false
- runtimeAssertionsBuild=false
- storageNodesVersion=v0.1.19-nightly.20260914
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21516296/ycsb-e-nodes-3-binary-basalt-storage-basalt-basalt-nodes-3-rsen/1789427998035/1789428152591)

_Logs were not uploaded to Datadog_

/cc @cockroachlabs/test-eng

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*ycsb/E/nodes=3/binary=basalt/storage=basalt/basalt_nodes=3/rsengine=enabled.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-68410

Start with the roachtest README and the artifacts under /artifacts/ycsb/E/nodes=3/binary=basalt/storage=basalt/basalt_nodes=3/rsengine=enabled/run_1. Inspect the fatal Pebble log at version_set.go:456 and compare it with the linked TeamCity build and Grafana run. Done means identifying the cause of the unsupported protocol version, addressing it, and confirming the roachtest passes.

Written by the indexing model from the issue text.