bazel-contrib / bazel-contrib/rules_go
go_tool_binary does not have reproducible output
- Dominant language
- Go
- Stars
- 1.5k
- Forks
- 762
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 12
Description
### What version of rules_go are you using?
Tested version between v0.23.8 and v0.25.1
### What version of gazelle are you using?
N/A
### What version of Bazel are you using?
3.7.1
### Does this issue reproduce with the latest releases of all the above?
Yes
### What operating system and processor architecture are you using?
darwin x86_64
### What did you do?
Built a go project.
### What did you expect to see?
Deterministic builds which leads to more remote-cache hits.
### What did you see instead?
Not deterministic go_sdk/builder leading to potential remote-cache misses.
I stumbled across this as I was debugging remote caching via: https://docs.bazel.build/versions/master/remote-execution-caching-debug.html#ensure-re-running-the-same-buildtest-command-produces-cache-hits
I'm not sure if the go_sdk/builder is actually causing my overall cache misses, but does seem like something that could cause problems.
[This PR](https://github.com/bazelbuild/rules_go/pull/2011) sent me down the objdump rathole and showed the sandbox paths in the built builder.
Have a standalone repo to show the potential problem:
https://github.com/connyay/rules_go_deterministic_builder_demo
```
bazel clean
INFO: Starting clean.
bazel build //cmd/main:main
INFO: Analyzed target //cmd/main:main (36 packages loaded, 6938 targets configured).
INFO: Found 1 target...
Target //cmd/main:main up-to-date:
bazel-bin/cmd/main/main_/main
INFO: Elapsed time: 4.948s, Critical Path: 2.75s
INFO: 8 processes: 4 internal, 4 darwin-sandbox.
INFO: Build completed successfully, 8 total actions
sha1sum bazel-out/host/bin/external/go_sdk/builder
ae6779e572aacd382e4caa4e981aafc7c9297d1c bazel-out/host/bin/external/go_sdk/builder
```
```
bazel clean
INFO: Starting clean.
bazel build //cmd/main:main
INFO: Analyzed target //cmd/main:main (36 packages loaded, 6938 targets configured).
INFO: Found 1 target...
Target //cmd/main:main up-to-date:
bazel-bin/cmd/main/main_/main
INFO: Elapsed time: 4.546s, Critical Path: 2.16s
INFO: 8 processes: 4 internal, 4 darwin-sandbox.
INFO: Build completed successfully, 8 total actions
sha1sum bazel-out/host/bin/external/go_sdk/builder
82495a996aeb2be6a43ff3df44d2699d3888830a bazel-out/host/bin/external/go_sdk/builder
```
I poked around a bit and couldnt pin down how the `go_sdk/src` is built. Not sure if there are work arounds by using the host sdk or something similar.
Contributor guide
Research direction
Reproduce the differing hashes in the linked standalone repository with `bazel clean` and `bazel build //cmd/main:main`. Start by tracing how `go_sdk/builder` and `go_sdk/src` are built, and inspect the builder with objdump; done means repeated clean builds produce identical builder output and reliable remote-cache hits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100