bazel-contrib / bazel-contrib/rules_go

Download prebuilt object files for the race detector.

Open
#3,258 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
1.5k
Forks
760
Avg merge
1d 11h
Merged PRs (30d)
12

Description

## The Problem

I tried to build a `go_binary` target with `cgo = True` and `race = "on"` from a macOS device, using the musl toolchain and targeting linux.

The build fails because the source for the race detector is strongly tied to `libc`, so some symbols are missing from `musl`.

Go ships with prebuilt versions of `race` that allows any sdk to cross-compile to any other sdk. [Ref](https://github.com/golang/go/tree/master/src/runtime/race).

In the rules_go workflow, this means that the prebuilt `syso` files need to be present in source when we compile the standard library, in [actions/stdlib.bzl](https://github.com/bazelbuild/rules_go/blob/master/go/private/actions/stdlib.bzl#L76).

However, the distributions of the go sdk that we download (e.g. from https://go.dev/dl/ ) are host-specific, and only have one of the precompiled `race` libraries.

Slack thread with full context: [link](https://bazelbuild.slack.com/archives/CDBP88Z0D/p1658324098787939)

## Possible Solutions

EDIT: I've asked upstream to the go maintainers, to see what they think the best way is: https://github.com/golang/go/issues/54174

We need to make sure the precompiled `race` `syso` files make it to the sodlib stage.
For that, I'd like to ask: Which of these is best?

- Download a full source distribution (22MB) to use as a basis for the `stdlib` (i.e. we would compile sodlib from here), from somewhere like https://go.dev/dl/go1.18.4.src.tar.gz, in addition to the current sdk download.
- Download a full source distribution, but only use that to copy over the `race` `syso` files to the sdk we download, then proceed just like we have.
- Download just the race files, one by one, from somewhere like [the GitHub mirror](https://github.com/golang/go/tree/master/src/runtime/race).
- This would minimise downloads, but it's also fairly unorthodox as the GitHub mirror is not really thought for releases.

---

## Other questions:

### What version of rules_go are you using?

I'm developing in `master`

### What version of gazelle are you using?

N/A

### What version of Bazel are you using?

5.2.0

### Does this issue reproduce with the latest releases of all the above?

Yep

### What operating system and processor architecture are you using?

macOS arm64, but repros with macOS x86.

### Any other potentially useful information about your toolchain?

We use musl to compile cgo binaries to linux from macOS.

### What did you do?

I tried to build a `go_binary` target with `cgo = True` and `race = "on"`, using the aforementioned musl toolchain and targeting linux.

The build fails because the source for the race detector is strongly tied to `libc`, so some symbols are missing from `musl`.

Contributor guide

Open the contributing guide

Research direction

Start by reading actions/stdlib.bzl and the linked Go runtime/race sources to understand how standard-library compilation currently obtains race detector objects. Compare the three proposed distribution approaches and the upstream Go issue, then define how the required target-specific syso files should reach the stdlib stage. Done means cross-compiling a cgo binary with race enabled through the musl toolchain succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
build-system, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.