cockroachdb / cockroachdb/cockroach
roachtest: c2c/shutdown/src/coordinator failed
- Ngôn ngữ chính
- Go
- Star
- 32.5k
- Fork
- 4.1k
- Chỉ số merge pull request
- Chỉ số pull request đang chờ
Mô tả
**Note:** This build has runtime assertions enabled. If the same failure was hit in a run without assertions enabled, there should be a similar failure without this message. If there isn't one, then this failure is likely due to an assertion violation or (assertion) timeout.
roachtest.c2c/shutdown/src/coordinator [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGcePrivateBazel/21516773?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGcePrivateBazel/21516773?buildTab=artifacts#/c2c/shutdown/src/coordinator) on master @ [b112dc518c826da8b4b577934d4ab5c0c24da045](https://github.com/cockroachdb/cockroach/commits/b112dc518c826da8b4b577934d4ab5c0c24da045):
Failed with:
```
(cluster_to_cluster.go:2128).func1: shutdown execution failed: could not restart node :3: failed to wait for SQL to be ready: open /tmp/roachtest-certs2416886761/certs/ca.crt: no such file or directory
(assertions.go:388).Fail:
Error Trace: pkg/cmd/roachtest/tests/cluster_to_cluster.go:1137
pkg/cmd/roachtest/tests/cluster_to_cluster.go:1408
pkg/cmd/roachtest/tests/cluster_to_cluster.go:2079
pkg/cmd/roachtest/monitor.go:295
external/org_golang_x_sync/errgroup/errgroup.go:93
src/runtime/asm_amd64.s:1771
Error: Received unexpected error:
failed stripped fingerprint: failed getting dst fingerprint: could not get database names: context canceled
(1) secondary error attachment
| already failed
| (1) attached stack trace
| -- stack trace:
| | main.(*monitorImpl).WaitE
| | pkg/cmd/roachtest/monitor.go:314
| | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*replicationDriver).onFingerprintMismatch
| | pkg/cmd/roachtest/tests/cluster_to_cluster.go:1136
| | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*replicationDriver).main
| | pkg/cmd/roachtest/tests/cluster_to_cluster.go:1408
| | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerClusterReplicationResilience.func1.3
| | pkg/cmd/roachtest/tests/cluster_to_cluster.go:2079
| | main.(*monitorImpl).Go.func1
| | pkg/cmd/roachtest/monitor.go:295
| | golang.org/x/sync/errgroup.(*Group).Go.func1
| | external/org_golang_x_sync/errgroup/errgroup.go:93
| | runtime.goexit
| | src/runtime/asm_amd64.s:1771
| Wraps: (2) already failed
| Error types: (1) *withstack.withStack (2) *errutil.leafError
Wraps: (2) failed stripped fingerprint
Wraps: (3) failed getting dst fingerprint
Wraps: (4) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/testutils/fingerprintutils.FingerprintAllDatabases
| pkg/testutils/fingerprintutils/fingerprint.go:178
| github.com/cockroachdb/cockroach/pkg/crosscluster/replicationutils.fingerprintClustersByTable.func2
| pkg/crosscluster/replicationutils/utils.go:284
| golang.org/x/sync/errgroup.(*Group).Go.func1
| external/org_golang_x_sync/errgroup/errgroup.go:93
| runtime.goexit
| src/runtime/asm_amd64.s:1771
Wraps: (5) could not get database names
Wraps: (6) context canceled
Error types: (1) *secondary.withSecondaryError (2) *fmt.wrapError (3) *fmt.wrapError (4) *withstack.withStack (5) *errutil.withPrefix (6) *errors.errorString
Test: c2c/shutdown/src/coordinator
(require.go:1376).NoError: FailNow called
test artifacts and logs in: /artifacts/c2c/shutdown/src/coordinator/run_1
```
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
| --- | --- | --- |
| teamcity-21516773-1789451745-32-n9cpu8-0001 | | 10.91.99.192 |
| teamcity-21516773-1789451745-32-n9cpu8-0002 | | 10.91.99.203 |
| teamcity-21516773-1789451745-32-n9cpu8-0003 | | 10.91.99.158 |
| teamcity-21516773-1789451745-32-n9cpu8-0004 | | 10.91.98.179 |
| teamcity-21516773-1789451745-32-n9cpu8-0005 | | 10.91.99.176 |
| teamcity-21516773-1789451745-32-n9cpu8-0006 | | 10.91.99.190 |
| teamcity-21516773-1789451745-32-n9cpu8-0007 | | 10.91.98.243 |
| teamcity-21516773-1789451745-32-n9cpu8-0008 | | 10.91.99.204 |
| teamcity-21516773-1789451745-32-n9cpu8-0009 | | 10.91.99.162 |
Parameters:
- arch=amd64
- cloud=gce
- coverageBuild=false
- cpu=8
- diskCount=0
- encrypted=false
- metamorphicLeases=expiration
- runtimeAssertionsBuild=true
Help
See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)
See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)
See: [Grafana](https://go.crdb.dev/roachtest-grafana/teamcity-21516773/c2c-shutdown-src-coordinator/1789465599141/1789466448143)
See: [Datadog Logs](https://us5.datadoghq.com/logs?query=service%3Aroachtest+%40cluster%3Ateamcity-21516773-1789451745-32-n9cpu8&from_ts=1789465599141&to_ts=1789466448143&live=false&storage=flex_tier)
/cc @cockroachlabs/disaster-recovery
[Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)
Jira issue: CRDB-68423
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.