bazel-contrib / bazel-contrib/rules_go

go_tool_binary does not have reproducible output

Open
#2,780 2 comments 0 reactions 0 assignees View on GitHub
bug cgo
Dominant language
Go
Stars
1.5k
Forks
762
Avg merge
1d 11h
Merged PRs (30d)
12

Description

### What version of rules_go are you using?

Tested version between v0.23.8 and v0.25.1

### What version of gazelle are you using?

N/A

### What version of Bazel are you using?

3.7.1

### Does this issue reproduce with the latest releases of all the above?

Yes

### What operating system and processor architecture are you using?

darwin x86_64

### What did you do?

Built a go project.

### What did you expect to see?

Deterministic builds which leads to more remote-cache hits.

### What did you see instead?

Not deterministic go_sdk/builder leading to potential remote-cache misses.

I stumbled across this as I was debugging remote caching via: https://docs.bazel.build/versions/master/remote-execution-caching-debug.html#ensure-re-running-the-same-buildtest-command-produces-cache-hits

I'm not sure if the go_sdk/builder is actually causing my overall cache misses, but does seem like something that could cause problems.

[This PR](https://github.com/bazelbuild/rules_go/pull/2011) sent me down the objdump rathole and showed the sandbox paths in the built builder.

Have a standalone repo to show the potential problem:
https://github.com/connyay/rules_go_deterministic_builder_demo

```
bazel clean
INFO: Starting clean.
bazel build //cmd/main:main
INFO: Analyzed target //cmd/main:main (36 packages loaded, 6938 targets configured).
INFO: Found 1 target...
Target //cmd/main:main up-to-date:
bazel-bin/cmd/main/main_/main
INFO: Elapsed time: 4.948s, Critical Path: 2.75s
INFO: 8 processes: 4 internal, 4 darwin-sandbox.
INFO: Build completed successfully, 8 total actions
sha1sum bazel-out/host/bin/external/go_sdk/builder
ae6779e572aacd382e4caa4e981aafc7c9297d1c bazel-out/host/bin/external/go_sdk/builder
```

```
bazel clean
INFO: Starting clean.
bazel build //cmd/main:main
INFO: Analyzed target //cmd/main:main (36 packages loaded, 6938 targets configured).
INFO: Found 1 target...
Target //cmd/main:main up-to-date:
bazel-bin/cmd/main/main_/main
INFO: Elapsed time: 4.546s, Critical Path: 2.16s
INFO: 8 processes: 4 internal, 4 darwin-sandbox.
INFO: Build completed successfully, 8 total actions
sha1sum bazel-out/host/bin/external/go_sdk/builder
82495a996aeb2be6a43ff3df44d2699d3888830a bazel-out/host/bin/external/go_sdk/builder
```

I poked around a bit and couldnt pin down how the `go_sdk/src` is built. Not sure if there are work arounds by using the host sdk or something similar.

Contributor guide

Open the contributing guide

Research direction

Reproduce the differing hashes in the linked standalone repository with `bazel clean` and `bazel build //cmd/main:main`. Start by tracing how `go_sdk/builder` and `go_sdk/src` are built, and inspect the builder with objdump; done means repeated clean builds produce identical builder output and reliable remote-cache hits.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.