bazel-contrib / bazel-contrib/rules_foreign_cc

gnumake/pkg-config toolchains are not reproducible

Open
#1,362 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Starlark
Stars
737
Forks
270
PR merge metrics
No merged PRs in 30d

Description

When evaluating the cache hit rates, we found a bunch of tools from rules_foreign_cc that are not reproducible and are causing cache misses.

The diff of execution log shows that, the same input produces different outputs:

```diff
spawn BootstrapPkgConfig @@rules_foreign_cc//toolchains/private:pkgconfig_tool_default differs, -spawn1 +spawn2:
&main.Spawn{
... // 3 identical fields
InputFiles: {&main.Directory{Path: "bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rul"..., Files: {&{Path: "bin/make", Digest: {Hash: "83a8c36b70875791558cc2da3121d47c57a162da6aaac47be52f996ff2b4f2cf", SizeBytes: 265288}}, &{Path: "include/gnumake.h", Digest: {Hash: "3e38df96688ba32938ece2070219684616bd157750c8ba5042ccb790a49dcacc", SizeBytes: 2913}}, &{Path: "share/info/make.info", Digest: {Hash: "f1c94cd2eaeacdd0825396e4f336701434380da93f8402b2eb64bd5249035a48", SizeBytes: 6518}}, &{Path: "share/info/make.info-1", Digest: {Hash: "50b1810947f3053d81dc1b5349369c31cc30c90d1bd8886f62ca9b360bbced55", SizeBytes: 301118}}, ...}}, &main.File{Path: "bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rul"..., Digest: {Hash: "fdd76db514ca7ce17ba131adf54e4ea4a65eb4bd9aef6b9f047887133d9f9000", SizeBytes: 2608}}, &main.File{Path: "bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rul"..., Digest: {Hash: "71ce8abeb9b324e04ccaf9505fbf15aab5d843f07c9c48824a256cf30cfad94e", SizeBytes: 1527}}, &main.File{Path: "external/llvm_toolchain/bin/cc_wrapper.sh", Digest: {Hash: "20d3e98de66875e5067fbc79003c4fb4e351b042da7e150e12e0556ebd84af48", SizeBytes: 6620}}, ...},
ToolFiles: {&main.Directory{Path: "bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rul"..., Files: {&{Path: "bin/make", Digest: {Hash: "83a8c36b70875791558cc2da3121d47c57a162da6aaac47be52f996ff2b4f2cf", SizeBytes: 265288}}, &{Path: "include/gnumake.h", Digest: {Hash: "3e38df96688ba32938ece2070219684616bd157750c8ba5042ccb790a49dcacc", SizeBytes: 2913}}, &{Path: "share/info/make.info", Digest: {Hash: "f1c94cd2eaeacdd0825396e4f336701434380da93f8402b2eb64bd5249035a48", SizeBytes: 6518}}, &{Path: "share/info/make.info-1", Digest: {Hash: "50b1810947f3053d81dc1b5349369c31cc30c90d1bd8886f62ca9b360bbced55", SizeBytes: 301118}}, ...}}, &main.File{Path: "bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rul"..., Digest: {Hash: "fdd76db514ca7ce17ba131adf54e4ea4a65eb4bd9aef6b9f047887133d9f9000", SizeBytes: 2608}}, &main.File{Path: "bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rul"..., Digest: {Hash: "71ce8abeb9b324e04ccaf9505fbf15aab5d843f07c9c48824a256cf30cfad94e", SizeBytes: 1527}}, &main.File{Path: "external/llvm_toolchain/bin/cc_wrapper.sh", Digest: {Hash: "20d3e98de66875e5067fbc79003c4fb4e351b042da7e150e12e0556ebd84af48", SizeBytes: 6620}}, ...},
Outputs: []main.FileLike{
&main.Directory{
Path: "bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rul"...,
Files: []*main.File{
&{
Path: "bin/arm-apple-darwin24.1.0-pkg-config",
Digest: main.Digest{
Hash: strings.Join({
- "fca472b0f36eee88bc5900e203780d179e5d18019bf904f0ad11ccc684e65942",
+ "6368a23ab04848a12caa9e770ffbcf9abb0fd5aef4a04208ea8ac3843fd7d2bb",
}, ""),
SizeBytes: 636392,
},
},
&{
Path: "bin/pkg-config",
Digest: main.Digest{
Hash: strings.Join({
- "fca472b0f36eee88bc5900e203780d179e5d18019bf904f0ad11ccc684e65942",
+ "6368a23ab04848a12caa9e770ffbcf9abb0fd5aef4a04208ea8ac3843fd7d2bb",
}, ""),
SizeBytes: 636392,
},
},
&{Path: "share/aclocal/pkg.m4", Digest: {Hash: "140b9a7bc1fa8730b62e74f8413af2bd15b53969a084fc4a041ba70b11e352f3", SizeBytes: 10247}},
&{Path: "share/doc/pkg-config/pkg-config-guide.html", Digest: {Hash: "abec943644a7c87d15bae2f5d5763488a592183ff35319732d44892a42a167c1", SizeBytes: 17832}},
&{Path: "share/man/man1/pkg-config.1", Digest: {Hash: "3591588e106c870d35f9da41bec0528a7e6577c45a97bb08766a33f1638a49f9", SizeBytes: 23611}},
},
},
&main.File{
Path: "bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rul"...,
Digest: main.Digest{
Hash: strings.Join({
- "4382c7005c073460b2c9032f6b5ce1de70d70b7686e3021473e959d1364422e",
+ "299b9fd99f6b847daaf3baa9de59d23f4b27f18ea01e9459d88222aea5193e5",
"a",
}, ""),
SizeBytes: 86548,
},
},
},
TargetLabel: "@@rules_foreign_cc//toolchains/private:pkgconfig_tool_default",
Mnemonic: "BootstrapPkgConfig",
... // 8 identical fields
}
```

After looking into the produced binaries, the diff is from the leaked sandbox path:

```shell
diff <(strings /Volumes/BazelCache/cache/cas/fc/fca472b0f36eee88bc5900e203780d179e5d18019bf904f0ad11ccc684e65942 | sort | uniq) <(strings /Volumes/BazelCache/cache/cas/63/6368a23ab04848a12caa9e770ffbcf9abb0fd5aef4a04208ea8ac3843fd7d2bb | sort | uniq)
230,231c230,231
< /private/var/tmp/_bazel_xiaoyishi/79c76c5b4224312419386af2e6f9c092/sandbox/darwin-sandbox/3217/execroot/__main__/bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rules_foreign_cc/toolchains/private/pkgconfig/lib
< /private/var/tmp/_bazel_xiaoyishi/79c76c5b4224312419386af2e6f9c092/sandbox/darwin-sandbox/3217/execroot/__main__/bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rules_foreign_cc/toolchains/private/pkgconfig/lib/pkgconfig:/private/var/tmp/_bazel_xiaoyishi/79c76c5b4224312419386af2e6f9c092/sandbox/darwin-sandbox/3217/execroot/__main__/bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rules_foreign_cc/toolchains/private/pkgconfig/share/pkgconfig
---
> /private/var/tmp/_bazel_xiaoyishi/79c76c5b4224312419386af2e6f9c092/sandbox/darwin-sandbox/4643/execroot/__main__/bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rules_foreign_cc/toolchains/private/pkgconfig/lib
> /private/var/tmp/_bazel_xiaoyishi/79c76c5b4224312419386af2e6f9c092/sandbox/darwin-sandbox/4643/execroot/__main__/bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rules_foreign_cc/toolchains/private/pkgconfig/lib/pkgconfig:/private/var/tmp/_bazel_xiaoyishi/79c76c5b4224312419386af2e6f9c092/sandbox/darwin-sandbox/4643/execroot/__main__/bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/external/rules_foreign_cc/toolchains/private/pkgconfig/share/pkgconfig
```

Contributor guide

Open the contributing guide

Research direction

No source files or tests are named. Start at the BootstrapPkgConfig action for @@rules_foreign_cc//toolchains/private:pkgconfig_tool_default and compare the generated binaries and execution logs shown in the issue. Done means repeated builds produce identical outputs without sandbox paths embedded in the binaries.

Written by the indexing model from the issue text.

Assessment

Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.