bazel-contrib / bazel-contrib/rules_go
Download prebuilt object files for the race detector.
- Dominant language
- Go
- Stars
- 1.5k
- Forks
- 760
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 12
Description
## The Problem
I tried to build a `go_binary` target with `cgo = True` and `race = "on"` from a macOS device, using the musl toolchain and targeting linux.
The build fails because the source for the race detector is strongly tied to `libc`, so some symbols are missing from `musl`.
Go ships with prebuilt versions of `race` that allows any sdk to cross-compile to any other sdk. [Ref](https://github.com/golang/go/tree/master/src/runtime/race).
In the rules_go workflow, this means that the prebuilt `syso` files need to be present in source when we compile the standard library, in [actions/stdlib.bzl](https://github.com/bazelbuild/rules_go/blob/master/go/private/actions/stdlib.bzl#L76).
However, the distributions of the go sdk that we download (e.g. from https://go.dev/dl/ ) are host-specific, and only have one of the precompiled `race` libraries.
Slack thread with full context: [link](https://bazelbuild.slack.com/archives/CDBP88Z0D/p1658324098787939)
## Possible Solutions
EDIT: I've asked upstream to the go maintainers, to see what they think the best way is: https://github.com/golang/go/issues/54174
We need to make sure the precompiled `race` `syso` files make it to the sodlib stage.
For that, I'd like to ask: Which of these is best?
- Download a full source distribution (22MB) to use as a basis for the `stdlib` (i.e. we would compile sodlib from here), from somewhere like https://go.dev/dl/go1.18.4.src.tar.gz, in addition to the current sdk download.
- Download a full source distribution, but only use that to copy over the `race` `syso` files to the sdk we download, then proceed just like we have.
- Download just the race files, one by one, from somewhere like [the GitHub mirror](https://github.com/golang/go/tree/master/src/runtime/race).
- This would minimise downloads, but it's also fairly unorthodox as the GitHub mirror is not really thought for releases.
---
## Other questions:
### What version of rules_go are you using?
I'm developing in `master`
### What version of gazelle are you using?
N/A
### What version of Bazel are you using?
5.2.0
### Does this issue reproduce with the latest releases of all the above?
Yep
### What operating system and processor architecture are you using?
macOS arm64, but repros with macOS x86.
### Any other potentially useful information about your toolchain?
We use musl to compile cgo binaries to linux from macOS.
### What did you do?
I tried to build a `go_binary` target with `cgo = True` and `race = "on"`, using the aforementioned musl toolchain and targeting linux.
The build fails because the source for the race detector is strongly tied to `libc`, so some symbols are missing from `musl`.
Contributor guide
Research direction
Start by reading actions/stdlib.bzl and the linked Go runtime/race sources to understand how standard-library compilation currently obtains race detector objects. Compare the three proposed distribution approaches and the upstream Go issue, then define how the required target-specific syso files should reach the stdlib stage. Done means cross-compiling a cgo binary with race enabled through the musl toolchain succeeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- build-system, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100