bazel-contrib / bazel-contrib/rules_go

building with race detection (`@io_bazel_rules_go//go/config:race`) is very slow especially if using Go binaries to generate code

Open
#3,218 12 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
1.5k
Forks
760
Avg merge
1d 11h
Merged PRs (30d)
12

Description

Over at Cockroach we are having [this issue](cockroachdb/cockroach#81314) where building a `go_test` with `--@io_bazel_rules_go//go/config:race` takes a very long time. The fact that the stdlib needs to be recompiled with race detection is a problem, but more relevant seems to be that building a target with race detection invalidates the cache for *all `go_binary`s used to produce that target*, including `go_binary`s that are built as `exec_tools` for a `genrule`, and causes all of them to be re-built with race detection. If you use Go code to generate other Go code (as we do), the problem is now twofold:

* previous builds of the `go_binary` for code generation cannot be reused, so the binary (and presumably all or most of its dependencies) needs to be rebuilt with race detection enabled (a slower build process in and of itself).
* because of the overhead imposed by the race detector, running the binary itself becomes much slower (potentially for no reason as I probably don't care about races in code generators, and probably only care about the code under test).

Since our code generators produce identical output regardless of whether race detection is enabled, it's very wasteful to re-build the code generators and then run a much slower version of the binary as part of the race build.

Presumably this behavior is OK by default, but there should be a workaround. I have tried updating the `go_binary` declaration for these code generating binaries to have `race = "off"` but it appears to have no effect -- setting the value on the command line overrides everything in `BUILD` files. It would be sufficient if I could have a way to specify that certain targets really do not need to be built with race detection despite the command-line flags.

### What version of rules_go are you using?

0.32

### What version of gazelle are you using?

0.25

### What version of Bazel are you using?

5.1.0

### Does this issue reproduce with the latest releases of all the above?

Yes

### What operating system and processor architecture are you using?

Darwin ARM64

### Any other potentially useful information about your toolchain?

(None)

### What did you do?

Tried setting `race = "off"` for code-generating `go_binary` target when building a target that depends on that target as an `exec_tool`.

### What did you expect to see?

Ideally the code-generating binary would not be built with race-detection especially if I set `race = "off"` manually.

### What did you see instead?

Code-generating binary is built with race detection, invalidating the cache and slowing the entire build down.

Contributor guide

Open the contributing guide

Research direction

Start with the go_binary declarations used as exec_tools for genrule and reproduce a race-enabled build using rules_go 0.32 and Bazel 5.1. Trace how the --@io_bazel_rules_go//go/config:race command-line setting reaches those binaries despite race = "off" in BUILD files; done means code generators can remain non-race builds, reuse their cache, and produce the same output during a race build.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.