cockroachdb / cockroachdb/cockroach
roachtest: validate GCP resource dependencies from the new default project
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Epic: https://github.com/cockroachdb/cockroach/issues/77644
Inventory and design: https://github.com/cockroachlabs/cockroach/pull/2930
## Goal
Ensure roachtests running in the new default GCE project, under its runner and VM identities, can continue to read, write, and manage all GCP resources they depend on.
This is a compatibility and migration-validation issue. It does not require every existing resource to move into the new project: retaining a resource in a shared or legacy project with appropriate cross-project access is acceptable.
## Scope
For each runtime dependency identified in the inventory, decide whether it remains shared, is recreated in the new project, or is replaced, and make the minimum configuration or code changes needed by roachtest. The important categories include:
- Read-only inputs such as fixture, corpus, artifact, and Jepsen buckets.
- Read/write storage such as scheduled-backup, long-TTL backup, CDC temporary-output, fixture-generation, DLQ, and artifact-failover buckets.
- Project-local APIs and resources such as Compute snapshots, Cloud Logging queries, project metadata, and Pub/Sub topics.
- Identity-bound resources such as the CDC service-account delegation chain, KMS keys, backup impersonation identities, and the runner and VM service accounts.
- Hard-coded legacy project, bucket, topic, and service-account names that need configuration or replacement.
Infrastructure and IAM resources that are already managed by Terraform should not be recreated here. This issue should validate that the resulting identities and bindings are sufficient and track only the necessary roachtest, roachprod, and CI configuration changes.
## Acceptance criteria
- [ ] Each runtime dependency in the inventory has an explicit disposition: retained with cross-project access, recreated, replaced, or confirmed unused.
- [ ] Runner-side and VM-side access are tested separately where both are used.
- [ ] Necessary project/resource names are configurable; affected hard-coded legacy references are removed or updated.
- [ ] CI variables and secret-backed resource references are updated and validated where applicable.
- [ ] Representative read-only fixture/artifact downloads succeed.
- [ ] Scheduled backup and representative read/write bucket operations succeed.
- [ ] CDC GCS, Pub/Sub, and service-account delegation paths succeed.
- [ ] KMS/assume-role backup coverage succeeds.
- [ ] Snapshot creation, reuse, listing, and cleanup succeed in the intended project.
- [ ] Any uncovered incompatibility is fixed here or tracked in a linked follow-up issue.
## Out of scope
- Provisioning the private GCE network and base IAM resources, which is handled by Terraform.
- Private-address selection and node connectivity, tracked by the address-mode implementation issues.
- Moving every shared resource into the new project when cross-project access is sufficient.
- The final full-suite canary and default-address-mode rollout.
Jira issue: CRDB-66433
Contributor guide
Research direction
Start with the linked inventory/design pull request and epic, then trace the roachtest, roachprod, runner, VM, and CI configuration references identified there. Validate each dependency and identity path separately, and consider the work done when the acceptance checks pass or every incompatibility is tracked in a follow-up issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- gcp, go
- Domain
- cloud, infrastructure, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100