cockroachdb / cockroachdb/cockroach
roachtest: ssh_problem failed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
roachtest.ssh_problem [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/19780278?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/19780278?buildTab=artifacts#/sql-stats/mixed-version) on release-25.2.1-rc @ [592da43f5cc80f32a8277aa90e09c7e3362ff53c](https://github.com/cockroachdb/cockroach/commits/592da43f5cc80f32a8277aa90e09c7e3362ff53c):
```
test sql-stats/mixed-version failed: (mixedversion.go:829).Run: mixed-version test failure while running step 138 (restart mixed-version-tenant-9fvrs server on node 1 with binary version release-25.2.1-rc): _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/sql-stats/mixed-version/run_1/ssh/ssh_113738.286037514_n1_run-start-script.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/sql-stats/mixed-version/run_1
```
Parameters:
- arch=amd64
- cloud=gce
- coverageBuild=false
- cpu=4
- encrypted=false
- fs=ext4
- localSSD=true
- metamorphicBufferedSender=false
- mvtDeploymentMode=separate-process
- mvtVersions=v23.2.25 → v24.1.18 → v24.2.10 → v24.3.13 → release-25.2.1-rc
- runtimeAssertionsBuild=false
- ssd=0
Help
See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)
See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)
See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-19780278/sql-stats-mixed-version/1748257880542/1748260126335)
Same failure on other branches
- #146861 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.1.7-rc]
- #146783 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.26-rc]
- #146715 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.3.14-rc]
- #146182 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.1.6-rc]
- #146027 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2.0-rc]
- #145029 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.18-rc]
- #144757 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.17-rc]
- #144607 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2.24-rc]
- #144541 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.1.5-rc]
- #144529 roachtest: ssh_problem failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.3.11-rc]
/cc @cockroachdb/test-eng
[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*ssh_problem.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Jira issue: CRDB-50980
Contributor guide
Research direction
Start with pkg/cmd/roachtest/README.md and the failure at mixedversion.go:829. Inspect the retained ssh -vvv log in /artifacts/sql-stats/mixed-version/run_1/ssh/ssh_113738.286037514_n1_run-start-script.log, the Grafana run, and the related ssh_problem issues. Done means establishing the recurring SSH failure's cause and documenting or implementing an appropriate resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- infrastructure, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100