cockroachdb / cockroachdb/cockroach

roachtest: replicate test fixtures in AWS and Azure

Open
#105,968 3 comments 0 reactions 0 assignees View on GitHub
A-testeng-foundations A-testing C-enhancement T-testeng
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

A number of roachtests have to be skipped outside of `GCE` owing to the fact that they rely on loading fixtures from a gcs bucket, e.g., `cockroach-fixtures`, `cockroach-corpus`; see below. Some of the buckets already exist in AWS, e.g., `cockroach-fixtures-us-east-2`. Thus, some of the currently skipped AWS runs could be enabled by simply updating the URLs. However, we need to ensure the buckets are replicated; this way, a test can be executed in any provider's environment. Furthermore, some tests may not use `workload fixtures load` and instead rely on cloud-specific data loading, e.g., `gsutil`.

**NOTE**: it's important to enforce large fixtures do not cross (cloud) region boundaries, so as to reduce the cost of egress. In GCE, `cockroach-fixtures` is a _multi-region_ bucket, hence no egress between any of the US regions. Conversely, in AWS, `cockroach-fixtures-us-east-2` is a _regional_ bucket since AWS doesn't support _multi_region_ buckets.

Ideally, data/fixtures loaded by a roachtest are lifted to a `TestSpec` which in conjunction with fixtures replication and `CloudSpec` would always default to a location which avoids egress.

```
find pkg/cmd/roachtest -name "*.go" |xargs grep -i "only in GCE"
pkg/cmd/roachtest/tests/schemachange.go: t.Skip("uses gs://cockroach-fixtures, available only in GCE")
pkg/cmd/roachtest/tests/mixed_version_decl_schemachange_compat.go: t.Skip("uses gsutil with gs://cockroach-corpus, available only in GCE")
pkg/cmd/roachtest/tests/tpc_utils.go: t.Skip("uses gs://cockroach-fixtures, available only in GCE")
pkg/cmd/roachtest/tests/backup.go: t.Skip("uses gs://cockroach-fixtures, available only in GCE")
pkg/cmd/roachtest/tests/import.go: t.Skip("uses gs://cockroach-fixtures, available only in GCE")
pkg/cmd/roachtest/tests/import.go: t.Skip("uses gs://cockroach-fixtures, available only in GCE")
pkg/cmd/roachtest/tests/import.go: t.Skip("uses gs://cockroach-fixtures, available only in GCE")
pkg/cmd/roachtest/tests/tpcdsvec.go: t.Skip("uses gs://cockroach-fixtures, available only in GCE")
pkg/cmd/roachtest/tests/sqlsmith.go: t.Skip("uses gs://cockroach-fixtures, available only in GCE")
pkg/cmd/roachtest/tests/copy.go: t.Skip("uses gs://cockroach-fixtures, available only in GCE")
pkg/cmd/roachtest/tests/mixed_version_backup.go: t.Skip("uses gs://cockroachdb-backup-testing, available only in GCE")
pkg/cmd/roachtest/tests/import_cancellation.go: t.Skip("uses gs://cockroach-fixtures, available only in GCE")
```

Jira issue: CRDB-29295

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.