basalt-26.3: roachtest: kv95/enc=false/nodes=3/cpu=32/binary=basalt/storage=basalt/basalt_nodes=3/rsengine=disabled failed

roachtest.kv95/enc=false/nodes=3/cpu=32/binary=basalt/storage=basalt/basalt_nodes=3/rsengine=disabled [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyBenchmarksAwsBasaltBazel/21516296?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyBenchmarksAwsBasaltBazel/21516296?buildTab=artifacts#/kv95/enc=false/nodes=3/cpu=32/binary=basalt/storage=basalt/basalt_nodes=3/rsengine=disabled) on basalt-26.3 @ [2327de0124eca58e8ea82a577a805766789a48e5](https://github.com/cockroachdb/cockroach/commits/2327de0124eca58e8ea82a577a805766789a48e5):

Failed with:

```
(cluster.go:4230).Start: COMMAND_PROBLEM: exit status 1
test artifacts and logs in: /artifacts/kv95/enc=false/nodes=3/cpu=32/binary=basalt/storage=basalt/basalt_nodes=3/rsengine=disabled/run_1
```
Fatal entries found in Cockroach logs:

```
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 MANIFEST flush failed: version 0: unsupported protocol version
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !goroutine 141 [running]:
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/allstacks.GetWithBuf({0x0?, 0x3f354c9ed798?, 0xd6afcb?})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/allstacks/allstacks.go:38 +0x125
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/allstacks.Get(...)
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/allstacks/allstacks.go:19
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/log.(*loggerT).outputLogEntry(_, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}}, ...})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/log/clog.go:294 +0xc6
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x86e51c8, 0x3f354ca343c0}, 0x3, 0x4, 0x3, 0x0?, {0x8630d28, 0x2}, {0x3f35495a5600, 0x1, ...})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/log/channels.go:104 +0x650
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 MANIFEST flush failed: version 0: unsupported protocol version
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !goroutine 141 [running]:
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/allstacks.GetWithBuf({0x0?, 0x3f354c9ed798?, 0xd6afcb?})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/allstacks/allstacks.go:38 +0x125
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/allstacks.Get(...)
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/allstacks/allstacks.go:19
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/log.(*loggerT).outputLogEntry(_, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}}, ...})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/log/clog.go:294 +0xc6
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 !github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x86e51c8, 0x3f354ca343c0}, 0x3, 0x4, 0x3, 0x0?, {0x8630d28, 0x2}, {0x3f35495a5600, 0x1, ...})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 8 ! pkg/util/log/channels.go:104 +0x650
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 MANIFEST flush failed: version 0: unsupported protocol version
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !goroutine 141 [running]:
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/allstacks.GetWithBuf({0x0?, 0x3f354c9ed798?, 0xd6afcb?})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/allstacks/allstacks.go:38 +0x125
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/allstacks.Get(...)
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/allstacks/allstacks.go:19
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/log.(*loggerT).outputLogEntry(_, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}}, ...})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/log/clog.go:294 +0xc6
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x86e51c8, 0x3f354ca343c0}, 0x3, 0x4, 0x3, 0x0?, {0x8630d28, 0x2}, {0x3f35495a5600, 0x1, ...})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/log/channels.go:104 +0x650
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 [n?,s?,pebble] 27 MANIFEST flush failed: version 0: unsupported protocol version
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 [n?,s?,pebble] 27 !goroutine 141 [running]:
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 [n?,s?,pebble] 27 !github.com/cockroachdb/cockroach/pkg/util/allstacks.GetWithBuf({0x0?, 0x3f354c9ed798?, 0xd6afcb?})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 [n?,s?,pebble] 27 ! pkg/util/allstacks/allstacks.go:38 +0x125
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 [n?,s?,pebble] 27 !github.com/cockroachdb/cockroach/pkg/util/allstacks.Get(...)
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 [n?,s?,pebble] 27 ! pkg/util/allstacks/allstacks.go:19
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 [n?,s?,pebble] 27 !github.com/cockroachdb/cockroach/pkg/util/log.(*loggerT).outputLogEntry(_, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}}, ...})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 [n?,s?,pebble] 27 ! pkg/util/log/clog.go:294 +0xc6
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 [n?,s?,pebble] 27 !github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x86e51c8, 0x3f354ca343c0}, 0x3, 0x4, 0x3, 0x0?, {0x8630d28, 0x2}, {0x3f35495a5600, 0x1, ...})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 [n?,s?,pebble] 27 ! pkg/util/log/channels.go:104 +0x650
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 MANIFEST flush failed: version 0: unsupported protocol version
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !goroutine 141 [running]:
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/allstacks.GetWithBuf({0x0?, 0x3f354c9ed798?, 0xd6afcb?})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/allstacks/allstacks.go:38 +0x125
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/allstacks.Get(...)
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/allstacks/allstacks.go:19
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/log.(*loggerT).outputLogEntry(_, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, ...}}, ...})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/log/clog.go:294 +0xc6
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 !github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x86e51c8, 0x3f354ca343c0}, 0x3, 0x4, 0x3, 0x0?, {0x8630d28, 0x2}, {0x3f35495a5600, 0x1, ...})
F260914 22:38:28.915709 141 3@pebble/version_set.go:456 ⋮ [n?,s?,pebble] 25 ! pkg/util/log/channels.go:104 +0x650
```
Cluster Node to Ip Mapping:

| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21516296-1789423542-10-n4cpu32-0001 | 3.133.130.19 | 10.12.5.131 |
| teamcity-21516296-1789423542-10-n4cpu32-0002 | 3.144.227.10 | 10.12.5.98 |
| teamcity-21516296-1789423542-10-n4cpu32-0003 | 18.216.188.112 | 10.12.2.35 |
| teamcity-21516296-1789423542-10-n4cpu32-0004 | 3.147.80.174 | 10.12.11.134 |

Parameters:
- MetamorphicBasaltMultiAZ=false
- MetamorphicFilesystem=ext4
- arch=amd64
- cloud=aws
- coverageBuild=false
- cpu=32
- diskCount=0
- encrypted=false
- fs=ext4
- localSSD=false
- runtimeAssertionsBuild=false
- storageNodesVersion=v0.1.19-nightly.20260914
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21516296/kv95-enc-false-nodes-3-cpu-32-binary-basalt-storage-basalt-basa/1789425497017/1789425650849)

_Logs were not uploaded to Datadog_

/cc @cockroachlabs/test-eng

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*kv95/enc=false/nodes=3/cpu=32/binary=basalt/storage=basalt/basalt_nodes=3/rsengine=disabled.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-68402

Start with the roachtest README and the failure at cluster.go:4230, then inspect the linked TeamCity artifacts and logs for the kv95 basalt run. Determine why Pebble reports an unsupported protocol version during MANIFEST flush, and verify that the same roachtest configuration starts successfully without fatal log entries.

Written by the indexing model from the issue text.