roachtest: ssh_problem failed

roachtest.ssh_problem [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/20734347?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/20734347?buildTab=artifacts#/failover/chaos/read-only/lease=leader) on release-25.4.1-rc @ [cdee83d12f6028cc8c2dd472e0df65d7bd74d72e](https://github.com/cockroachdb/cockroach/commits/cdee83d12f6028cc8c2dd472e0df65d7bd74d72e):

Failed with:

```
test failover/chaos/read-only/lease=leader failed: (cluster.go:2436).Stop: cluster.StopE: one or more parallel execution failure(s): _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/failover/chaos/read-only/lease=leader/cpu_arch=arm64/run_1/ssh/ssh_112812.463460133_n2_kill.log): TRANSIENT_ERROR(ssh_problem): exit status 255
(cluster.go:2501).Run: context canceled
(cluster.go:2501).Run: context canceled
(cluster.go:2501).Run: context canceled
test artifacts and logs in: /artifacts/failover/chaos/read-only/lease=leader/cpu_arch=arm64/run_1
```
Fatal entries found in Cockroach logs:

```
F251114 11:20:17.194868 175582 storage/pebble.go:1361 ⋮ [n6,s6,pebble] 869 disk stall detected: disk slowness detected: syncdata on file 001079.log has been ongoing for 20.4s
F251114 11:20:17.194868 175582 storage/pebble.go:1361 ⋮ [n6,s6,pebble] 869 +‹Time Device Name Reads Completed Reads Merged Sectors Read Read Duration Writes Completed Writes Merged Sectors Written Write Duration IO in Progress IO Duration Weighted IO Duration Discards Completed Discards Merged Sectors Discarded Discard Duration Flushes Completed Flush Duration Error›
F251114 11:20:17.194868 175582 storage/pebble.go:1361 ⋮ [n6,s6,pebble] 869 +‹2025-11-14T11:19:47.320374483Z dm-0 0 0 0 0s 4 0 16 4ms 0 12ms 4ms 0 0 0 0s 0 0s nil›
F251114 11:20:17.194868 175582 storage/pebble.go:1361 ⋮ [n6,s6,pebble] 869 +‹2025-11-14T11:19:47.419748748Z dm-0 0 0 0 0s 6 0 728 4ms 0 12ms 4ms 0 0 0 0s 0 0s nil›
F251114 11:20:17.194868 175582 storage/pebble.go:1361 ⋮ [n6,s6,pebble] 869 +‹2025-11-14T11:19:47.520612773Z dm-0 0 0 0 0s 2 0 8 0s 0 4ms 0s 0 0 0 0s 0 0s nil›
F251114 11:20:17.194868 175582 storage/pebble.go:1361 ⋮ [n6,s6,pebble] 869 +‹2025-11-14T11:19:47.620828359Z dm-0 0 0 0 0s 0 0 0 0s 0 0s 0s 0 0 0 0s 0 0s nil›
F251114 11:20:17.194868 175582 storage/pebble.go:1361 ⋮ [n6,s6,pebble] 869 +‹2025-11-14T11:19:47.720576264Z dm-0 0 0 0 0s 0 0 0 0s 0 0s 0s 0 0 0 0s 0 0s nil›
F251114 11:20:17.194868 175582 storage/pebble.go:1361 ⋮ [n6,s6,pebble] 869 +‹2025-11-14T11:19:47.819978769Z dm-0 0 0 0 0s 2 0 8 0s 0 4ms 0s 0 0 0 0s 0 0s nil›
F251114 11:20:17.194868 175582 storage/pebble.go:1361 ⋮ [n6,s6,pebble] 869 +‹2025-11-14T11:19:47.919773555Z dm-0 0 0 0 0s 2 0 8 4ms 0 8ms 4ms 0 0 0 0s 0 0s nil›
F251114 11:20:17.194868 175582 storage/pebble.go:1361 ⋮ [n6,s6,pebble] 869 +‹2025-11-14T11:19:48.02003494Z dm-0 0 0 0 0s 2 0 8 0s 0 4ms 0s 0 0 0 0s 0 0s nil›
F251114 11:01:24.709179 121581 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 1131 disk stall detected: disk slowness detected: syncdata on file 000274.log has been ongoing for 20.5s
F251114 11:01:24.709179 121581 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 1131 +‹Time Device Name Reads Completed Reads Merged Sectors Read Read Duration Writes Completed Writes Merged Sectors Written Write Duration IO in Progress IO Duration Weighted IO Duration Discards Completed Discards Merged Sectors Discarded Discard Duration Flushes Completed Flush Duration Error›
F251114 11:01:24.709179 121581 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 1131 +‹2025-11-14T11:00:54.832287992Z dm-0 0 0 0 0s 18 0 96 0s 0 12ms 0s 0 0 0 0s 0 0s nil›
F251114 11:01:24.709179 121581 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 1131 +‹2025-11-14T11:00:54.932609145Z dm-0 0 0 0 0s 22 0 120 8ms 0 20ms 8ms 0 0 0 0s 0 0s nil›
F251114 11:01:24.709179 121581 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 1131 +‹2025-11-14T11:00:55.031787618Z dm-0 0 0 0 0s 0 0 0 0s 0 0s 0s 0 0 0 0s 0 0s nil›
F251114 11:01:24.709179 121581 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 1131 +‹2025-11-14T11:00:55.13210013Z dm-0 0 0 0 0s 18 0 96 4ms 0 16ms 4ms 0 0 0 0s 0 0s nil›
F251114 11:01:24.709179 121581 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 1131 +‹2025-11-14T11:00:55.231873443Z dm-0 0 0 0 0s 13 0 72 0s 0 16ms 0s 0 0 0 0s 0 0s nil›
F251114 11:01:24.709179 121581 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 1131 +‹2025-11-14T11:00:55.332591236Z dm-0 0 0 0 0s 9 0 48 0s 0 8ms 0s 0 0 0 0s 0 0s nil›
F251114 11:01:24.709179 121581 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 1131 +‹2025-11-14T11:00:55.432452829Z dm-0 0 0 0 0s 0 0 0 0s 0 0s 0s 0 0 0 0s 0 0s nil›
F251114 11:01:24.709179 121581 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 1131 +‹2025-11-14T11:00:55.531751501Z dm-0 0 0 0 0s 9 0 48 8ms 0 16ms 8ms 0 0 0 0s 0 0s nil›
F251114 11:16:04.352601 171248 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 2571 disk stall detected: disk slowness detected: syncdata on file 000336.log has been ongoing for 21.7s
F251114 11:16:04.352601 171248 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 2571 +‹Time Device Name Reads Completed Reads Merged Sectors Read Read Duration Writes Completed Writes Merged Sectors Written Write Duration IO in Progress IO Duration Weighted IO Duration Discards Completed Discards Merged Sectors Discarded Discard Duration Flushes Completed Flush Duration Error›
F251114 11:16:04.352601 171248 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 2571 +‹2025-11-14T11:15:34.521134142Z dm-0 0 0 0 0s 8 0 32 4ms 0 16ms 4ms 0 0 0 0s 0 0s nil›
F251114 11:16:04.352601 171248 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 2571 +‹2025-11-14T11:15:34.621258507Z dm-0 0 0 0 0s 0 0 0 0s 0 0s 0s 0 0 0 0s 0 0s nil›
F251114 11:16:04.352601 171248 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 2571 +‹2025-11-14T11:15:34.721993512Z dm-0 0 0 0 0s 4 0 16 0s 0 4ms 0s 0 0 0 0s 0 0s nil›
F251114 11:16:04.352601 171248 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 2571 +‹2025-11-14T11:15:34.821450637Z dm-0 0 0 0 0s 0 0 0 0s 0 0s 0s 0 0 0 0s 0 0s nil›
F251114 11:16:04.352601 171248 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 2571 +‹2025-11-14T11:15:34.921371881Z dm-0 0 0 0 0s 2 0 8 0s 0 4ms 0s 0 0 0 0s 0 0s nil›
F251114 11:16:04.352601 171248 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 2571 +‹2025-11-14T11:15:35.021325086Z dm-0 0 0 0 0s 4 0 16 0s 0 4ms 0s 0 0 0 0s 0 0s nil›
F251114 11:16:04.352601 171248 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 2571 +‹2025-11-14T11:15:35.121733371Z dm-0 0 0 0 0s 8 0 32 0s 0 8ms 0s 0 0 0 0s 0 0s nil›
F251114 11:16:04.352601 171248 storage/pebble.go:1361 ⋮ [n8,s8,pebble] 2571 +‹2025-11-14T11:15:35.221286776Z dm-0 0 0 0 0s 18 0 80 4ms 0 16ms 4ms 0 0 0 0s 0 0s nil›
```

Parameters:
- arch=arm64
- cloud=gce
- coverageBuild=false
- cpu=2
- encrypted=false
- fs=ext4
- localSSD=false
- runtimeAssertionsBuild=false
- ssd=0
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-20734347/failover-chaos-read-only-lease-leader/1763117020543/1763119958003)

Same failure on other branches

- #156988 roachtest: ssh_problem failed [O-roachtest O-robot O-rsg T-testeng X-infra-flake branch-release-25.3.5-rc]
- #156930 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2.9-rc]
- #156925 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.3.23-rc]
- #155684 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.4.0-rc]
- #155430 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2.7-rc]
- #155142 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3.3-rc]
- #154854 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.25-rc]
- #154612 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.3.21-rc]
- #154528 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.4]
- #153154 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.24-rc]

/cc @cockroachdb/test-eng

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*ssh_problem.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-56962