cockroachdb / cockroachdb/cockroach

roachtest: `Cluster.Start` should require `TestSpec`

Open
#139,525 2 comments 0 reactions 0 assignees View on GitHub
A-testeng-foundations C-enhancement T-testeng
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Currently, `Cluster.Start` is unaware of `TestSpec`. This reflects the historical fact that the `roachtest` framework doesn't manage how and when clusters are started. Each test determines when and how to invoke `Cluster.Start` inside its `Run` method.

In contrast, the `mixedversion` framework controls how clusters are started via `startStep`; e.g., individual mixedversion tests can customized cluster settings via `mixedversion.ClusterSettingOption`. The framework, in theory, could decide to override a specific setting. This is currently not possible in the `roachtest` proper.

The real impetus for augmenting `Cluster.Start` is discussed in [1]. A subset of roachtests are _performance_ tests, i.e., `TestSpec.Benchmark==true`. A sane default for the majority of performance tests should be the `NoBackupSchedule` start option. Thus, with the proposed change, the framework can enforce this default rather than relying on the test author to do the "right" thing.

[1] https://cockroachlabs.slack.com/archives/C023S0V4YEB/p1734972581281769

Jira issue: CRDB-46696

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.