cockroachdb / cockroachdb/cockroach

roachtest: add ability to execute tests with a binary built with `--race`

Open
#143,550 1 comment 0 reactions 0 assignees View on GitHub
A-testeng-foundations C-enhancement T-testeng
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

CLI code is currently covered via Tcl scripts which run inside Docker (see `runTestDockerCLI`). It doesn't appear they use a cockroach binary which was built with `--race`. Thus, data races inside CLI have fairly poor coverage since unit tests don't typically cover CLI.

A recent example illustrates the issue [1]. The PR introduced a data race in `pkg/workload/cli/run.go`. It passed CI and got merged. The problem wasn't detected until the nightly runs, where a relatively small number of roachtests running the workload ended up failing on concurrent map access. It turns out this data race is easily caught by simply invoking the workload via CLI, using a binary that was built with `--race`,

```
./cockroach_race workload run kv --ramp=1s --duration=1s

==================
WARNING: DATA RACE
Write at 0x00c001444528 by goroutine 154:
github.com/cockroachdb/cockroach/pkg/workload/cli.runRun.func5()
pkg/workload/cli/run.go:543 +0xec
```

Thus, we should consider running some of the "acceptance" roachtests with `--race`; the overhead shouldn't be too bad wrt CI time. Similarly, a subset of the Tcl scripts could be executed with `--race`.

[1] https://github.com/cockroachdb/cockroach/pull/142258

Jira issue: CRDB-48905

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.