cockroachdb / cockroachdb/cockroach

basalt-26.3: roachtest: cluster_creation failed

Open
#172,904 1355 comments 0 reactions 0 assignees View on GitHub
branch-basalt-26.3 O-roachtest O-robot T-testeng X-infra-flake
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

roachtest.cluster_creation [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGceBazel_Basalt/21444536?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_Roachtests_RoachtestNightlyGceBazel_Basalt/21444536?buildTab=artifacts#/acceptance/gossip/restart) on basalt-26.3 @ [e8a528937626edd5449fc9f54a0645ca2ee1bed3](https://github.com/cockroachdb/cockroach/commits/e8a528937626edd5449fc9f54a0645ca2ee1bed3):

```
vmType: c4a-standard-4-lssd
zone: us-east1-c
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A c4a-standard-4-lssd VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-c zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: c4a-standard-4-lssd
zone: us-east1-c
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
resources available to fulfill the request. '(resource type:compute)'.
---
code: ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS
errorDetails:
- help:
links:
- description: Troubleshooting documentation
url: https://cloud.google.com/compute/docs/resource-error
- localizedMessage:
locale: en-US
message: A c4a-standard-4-lssd VM instance with 1 local SSD(s) is currently unavailable
in the us-east1-c zone. Try requesting the VM in another zone. For more information,
view the troubleshooting documentation.
- errorInfo:
domain: compute.googleapis.com
metadatas:
attachment: local-ssd:1
vmType: c4a-standard-4-lssd
zone: us-east1-c
zonesAvailable: ''
reason: stockout
message: The zone 'projects/cockroach-ephemeral/zones/us-east1-c' does not have enough
resources available to fulfill the request. '(resource type:compute)'.: exit status 1 [owner=test-eng]
```

Parameters:
- cloud=gce
- coverageBuild=false
- cpu=4
- diskCount=0
- runtimeAssertionsBuild=false
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #172635 release-26.2: roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.2]
- #172559 release-26.3.0-rc: roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.3.0-rc]
- #172510 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.1.8-rc]
- #172509 release-26.2.5-rc: roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.2.5-rc]
- #172508 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.1.8-rc]
- #172507 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.4.14-rc]
- #172506 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2.22-rc]
- #172191 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.4]
- #172020 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2]
- #171764 release-26.3: roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.3]

/cc @cockroachlabs/test-eng

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*cluster_creation.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-66187

Contributor guide

Open the contributing guide

Research direction

Start with the roachtest README, then inspect the linked TeamCity log and artifacts for the cluster_creation failure. Compare the repeated cross-branch reports listed in the issue. Done means establishing an actionable cause or disposition for the GCE resource error and preventing this failure from continuing to generate the report.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, go
Domain
cloud, infrastructure, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.