cockroachdb / cockroachdb/cockroach
release-26.4: roachtest: replicate/wide failed
Open
Nobody has claimed this yet.
branch-release-26.4
C-test-failure
O-roachtest
O-robot
release-blocker
T-kv
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
roachtest.replicate/wide failed with artifacts on release-26.4 @ b4884d371d74d172ab184bc4ef0924ccb758f3c2:
Failed with:
(assertions.go:388).Fail:
Error Trace: pkg/cmd/roachtest/tests/allocator.go:458
pkg/cmd/roachtest/tests/allocator.go:464
pkg/cmd/roachtest/test_runner.go:1992
src/runtime/asm_amd64.s:1771
Error: Received unexpected error:
roachtest node 9 is not a CRDB node
(1) attached stack trace
-- stack trace:
| main.validateCockroachPhysicalNode
| pkg/cmd/roachtest/cluster_node_identities.go:112
| main.cockroachNode
| pkg/cmd/roachtest/cluster_node_identities.go:198
| main.(*roachprodCluster).CockroachNode
| pkg/cmd/roachtest/cluster_node_identities.go:229
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runWideReplication.func8
| pkg/cmd/roachtest/tests/allocator.go:457
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runWideReplication
| pkg/cmd/roachtest/tests/allocator.go:464
| main.(*testRunner).runTest.func2
| pkg/cmd/roachtest/test_runner.go:1992
| runtime.goexit
| src/runtime/asm_amd64.s:1771
Wraps: (2) roachtest node 9 is not a CRDB node
Error types: (1) *withstack.withStack (2) *errutil.leafError
Test: replicate/wide
(require.go:1376).NoError: FailNow called
test artifacts and logs in: /artifacts/replicate/wide/run_1
Cluster Node to Ip Mapping:
| Node | Public IP | Private IP |
|---|---|---|
| teamcity-21523842-1789803587-55-n9cpu1-0001 | 10.91.98.75 | |
| teamcity-21523842-1789803587-55-n9cpu1-0002 | 10.91.99.235 | |
| teamcity-21523842-1789803587-55-n9cpu1-0003 | 10.91.99.36 | |
| teamcity-21523842-1789803587-55-n9cpu1-0004 | 10.91.98.98 | |
| teamcity-21523842-1789803587-55-n9cpu1-0005 | 10.91.98.76 | |
| teamcity-21523842-1789803587-55-n9cpu1-0006 | 10.91.97.46 | |
| teamcity-21523842-1789803587-55-n9cpu1-0007 | 10.91.98.62 | |
| teamcity-21523842-1789803587-55-n9cpu1-0008 | 10.91.98.110 | |
| teamcity-21523842-1789803587-55-n9cpu1-0009 | 10.91.98.24 |
Parameters:
arch=amd64cloud=gcecoverageBuild=falsecpu=1diskCount=0encrypted=falsefs=ext4localSSD=truemetamorphicLeases=defaultruntimeAssertionsBuild=false
Same failure on other branches
- #175650 roachtest: replicate/wide failed [C-test-failure O-roachtest O-robot T-kv branch-master release-blocker]
Jira issue: CRDB-68556
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with pkg/cmd/roachtest/tests/allocator.go at lines 457-464, then trace node validation through cluster_node_identities.go at lines 112, 198, and 229. Review the linked TeamCity artifacts and logs for replicate/wide and compare the related failure in #175650. Done means replicate/wide no longer reports node 9 as a non-CRDB node on the affected branches.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100