cockroachdb / cockroachdb/cockroach
roachtest: run import tests on AWS with S3 fixtures
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
[Technical advisory a161317](https://www.cockroachlabs.com/docs/advisories/a161317) (silent data loss in AVRO OCF import, [#161317](https://github.com/cockroachdb/cockroach/issues/161317)) identified a gap in storage backend test coverage. The existing import roachtests (`pkg/cmd/roachtest/tests/import.go`) only use GCS fixtures (`gs://cockroach-fixtures-us-east1/tpch-csv/`), so the S3 storage path is never exercised end-to-end by import tests.
We should get the import roachtests running on the existing AWS roachtest runs. This requires:
- Uploading the TPC-H fixtures (and AVRO fixtures from #164461, if available) to an S3 bucket
- Teaching the import roachtest how to discover the correct fixture URLs based on which cloud it's running in (GCS vs S3)
Jira issue: CRDB-60824
Epic CRDB-62881
Contributor guide
Assessment
This issue has not been assessed yet.