cockroachdb / cockroachdb/cockroach

roachtest: cluster_creation failed

Open
#168,567 239 comments 0 reactions 0 assignees View on GitHub
branch-release-25.4.10-rc O-roachtest O-robot T-testeng X-infra-flake
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

roachtest.cluster_creation [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyAwsBazel/21317058?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestNightlyAwsBazel/21317058?buildTab=artifacts#/kv0/enc=false/nodes=3/size=4kb) on release-25.4.10-rc @ [ee6fd101707e06e673804fa8cffae268a230bdb5](https://github.com/cockroachdb/cockroach/commits/ee6fd101707e06e673804fa8cffae268a230bdb5):

```
test kv0/enc=false/nodes=3/size=4kb failed: (test_runner.go:885).func4: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m6id.2xlarge --image-id ami-0e83be366243f524a --key-name teamcity-runner-0ZqyiEWoTkRFPc5xC9CBjY7RzeA= --region us-east-2 --security-group-ids sg-04d72b57e29d671f1 --subnet-id subnet-0258dc9d1b6473d84 --tag-specifications ResourceType=instance,Tags=[{Key=usage,Value=roachtest},{Key=roachtest-branch,Value=release-25-4-10-rc},{Key=Name,Value=teamcity-21317058-1776405329-33-n4cpu8-0002},{Key=Cluster,Value=teamcity-21317058-1776405329-33-n4cpu8},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-21317058-1776405329-33-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2026-04-17T06:42:15Z}] ResourceType=volume,Tags=[{Key=usage,Value=roachtest},{Key=roachtest-branch,Value=release-25-4-10-rc},{Key=Name,Value=teamcity-21317058-1776405329-33-n4cpu8-0002},{Key=Cluster,Value=teamcity-21317058-1776405329-33-n4cpu8},{Key=spot,Value=true},{Key=roachprod,Value=true},{Key=cluster,Value=teamcity-21317058-1776405329-33-n4cpu8},{Key=lifetime,Value=12h0m0s},{Key=arch,Value=amd64},{Key=created,Value=2026-04-17T06:42:15Z}] --user-data file:///tmp/aws-startup-script1714004700 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping637935760 --instance-market-options MarketType=spot,SpotOptions={SpotInstanceType=one-time,InstanceInterruptionBehavior=terminate} --output json: stderr:
An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient m6id.2xlarge capacity in the Availability Zone you requested (us-east-2a). Our system will be working on provisioning additional capacity. You can currently get m6id.2xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2b, us-east-2c.: exit status 254 [owner=test-eng]
```

Parameters:
- cloud=aws
- coverageBuild=false
- cpu=8
- runtimeAssertionsBuild=false
- ssd=0
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #168565 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2.18-rc]
- #168562 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.3.32-rc]
- #168445 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.1.3-rc]
- #168443 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-25.2.17-rc]
- #168440 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.3.31-rc]
- #167708 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.1]
- #167706 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.1]
- #167707 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-26.1]
- #167462 roachtest: cluster_creation failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

/cc @cockroachdb/test-eng

[This test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*cluster_creation.*&sort=title+created&display=lastcommented+project) | [Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-63008

Contributor guide

Open the contributing guide

Research direction

Start with the linked TeamCity failure log and artifacts, then read the roachtest README and inspect the related failures listed in this issue. Use the roachdash query to determine whether cluster_creation failures persist across branches; done means the AWS cluster creation failure is addressed or its expected handling is documented and verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go
Domain
cloud, infrastructure, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.