cockroachdb / cockroachdb/cockroach
roachtest: expand select-probability functionality
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
In #105388, we implemented a roachtest CLI parameter to allow users to specify a sampling rate of roachtests. This has allow new features to be added and smoke tested.
The initial implementation ensures that at least one test per prefix is selected, regardless of how small a probability is set, which guarantees solid coverage across a range of tests.
However, the caveat is that even the smallest probability will still run potentially > 100 tests (1 per prefix), which can take hours to run.
We should expose more parameters that allow:
- alternative grouping by owning team
- using strict sampling without minimum per group requirements
e.g.
- `--select-probability 0.1 would return 10% of all tests (after filtering)
- `--select-probability 0.1 --select-group owner --select-group-min ` would return at least 10% of all tests, and a minimum of per owner. (We could also choose to omit allowing to specify a number, in favour of `--select-group-atleast-one`)
Jira issue: CRDB-32103
Contributor guide
Assessment
This issue has not been assessed yet.