cockroachdb / cockroachdb/cockroach

release-26.2.3-rc: roachtest: schema_change_workload_failure failed

Open
#171,761 6 comments 0 reactions 0 assignees View on GitHub
branch-release-26.2.3-rc C-bug O-roachtest O-robot T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

roachtest.schema_change_workload_failure [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/21392007?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyGceBazel/21392007?buildTab=artifacts#/backup-restore/online-restore) on release-26.2.3-rc @ [766dac47934d29a38722ddfe951ef663cf6429ec](https://github.com/cockroachdb/cockroach/commits/766dac47934d29a38722ddfe951ef663cf6429ec):

Failed with:

```
test backup-restore/online-restore failed: (test_runner.go:2716).func1: task ``run-schemachange`` returned error: schema change workload failed: full command output in run_062532.930232723_n4_COCKROACHRANDOMSEED5.log: COMMAND_PROBLEM: exit status 1 [owner=sql-foundations]
(monitor.go:267).Wait: monitor failure: context canceled
test artifacts and logs in: /artifacts/backup-restore/online-restore/run_1
```
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21392007-1781763042-12-n4cpu4-0001 | 34.75.241.55 | 10.142.2.220 |
| teamcity-21392007-1781763042-12-n4cpu4-0002 | 34.75.236.212 | 10.142.2.219 |
| teamcity-21392007-1781763042-12-n4cpu4-0003 | 35.231.156.65 | 10.142.2.223 |
| teamcity-21392007-1781763042-12-n4cpu4-0004 | 35.229.63.93 | 10.142.3.10 |

Parameters:
- arch=amd64
- cloud=gce
- coverageBuild=false
- cpu=4
- diskCount=0
- encrypted=true
- runtimeAssertionsBuild=false
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21392007/backup-restore-online-restore/1781763742700/1781764200762)

/cc @cockroachdb/sql-foundations

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*schema_change_workload_failure.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-64897

Contributor guide

Open the contributing guide

Research direction

Start with pkg/cmd/roachtest/README.md and the run-schemachange output in run_062532.930232723_n4_COCKROACHRANDOMSEED5.log under the online-restore artifacts. Review the Grafana run and the reported monitor failure to identify the cause. Done means backup-restore/online-restore completes without the schema change workload failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.