cockroachdb / cockroachdb/cockroach
roachtest: stage_failure failed
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
roachtest.stage_failure [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/20516654?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/20516654?buildTab=artifacts#/backup-restore/mixed-version) on release-25.4 @ [9221c5318324b33455257088d3a48fae69346359](https://github.com/cockroachdb/cockroach/commits/9221c5318324b33455257088d3a48fae69346359):
```
test backup-restore/mixed-version failed: (mixedversion.go:862).func2: cluster.Stage: output:: TRANSIENT_ERROR(stage_failure): context canceled
(mixedversion.go:889).Run: mixed-version test failure while running step 8 (run "initialize tpcc workload"): full command output in run_080638.118751550_n5_cockroach-workload-i.log: COMMAND_PROBLEM: exit status 4
test artifacts and logs in: /artifacts/backup-restore/mixed-version/run_1
```
Parameters:
- arch=amd64
- cloud=gce
- coverageBuild=false
- cpu=4
- encrypted=true
- fs=ext4
- localSSD=true
- mvtDeploymentMode=system-only
- mvtVersions=v25.2.5 → v25.3.1 → release-25.4
- runtimeAssertionsBuild=false
- ssd=0
Help
See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)
See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)
See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-20516654/backup-restore-mixed-version/1758787556243/1758787790015)
Same failure on other branches
- #151910 roachtest: stage_failure failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.3]
- #146357 roachtest: stage_failure failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.3]
- #146037 roachtest: stage_failure failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2.0-rc]
- #140132 roachtest: stage_failure failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.1]
- #123276 roachtest: stage_failure failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-23.2]
- #123193 roachtest: stage_failure failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]
- #123047 roachtest: stage_failure failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1]
/cc @cockroachdb/test-eng
[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*stage_failure.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Jira issue: CRDB-54737
Contributor guide
Research direction
Start with the roachtest README and the failure in mixedversion.go around steps 8 and cluster.Stage, then inspect run_080638.118751550_n5_cockroach-workload-i.log under the linked artifacts. Use the Grafana trace and related issues to determine the cause of the context cancellation; done means identifying and addressing the recurring stage_failure or documenting why it is expected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100