bazel-contrib / bazel-contrib/rules_foreign_cc

Make and pkg-config toolchains are not reproducible

Open
#1,313 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Starlark
Stars
737
Forks
270
PR merge metrics
No merged PRs in 30d

Description

When using (a [recent](https://github.com/bazel-contrib/rules_foreign_cc/commit/77d4483fadbb1b7bcace18ed8e8e87e8791050f6)) `rules_foreign_cc` I'm getting determinism issues in the toolchains. In my case the checksum of both the make and pkgconfig binaries change every time they're built:

```
$ bazel build //my_c_target
...
$ sha256sum bazel-out/k8-opt-exec-ST-d57f47055a04/bin/external/rules_foreign_cc~/toolchains/private/make/bin/make
e00a81041a33af3ed7885449b88129c9f83661bcbec6cc84e51c73f77d93340f bazel-out/k8-opt-exec-ST-d57f47055a04/bin/external/rules_foreign_cc~/toolchains/private/make/bin/make
$ bazel clean && bazel build //my_c_target
$ sha256sum bazel-out/k8-opt-exec-ST-d57f47055a04/bin/external/rules_foreign_cc~/toolchains/private/make/bin/make
95bf6a1b30b77689370e4e73a6f85a2684a6522db36d54d2744093d30532acb9 bazel-out/k8-opt-exec-ST-d57f47055a04/bin/external/rules_foreign_cc~/toolchains/private/make/bin/make
```

Note how each build produced a different `make` executable.

I've noticed that the build logs (`Configure.log`, `BootstrapGNUMake.log`, etc) introduce non-determinism too as they include full sandbox paths in their outputs (in commands and in "Can not copy X" lines).

The workaround for me is to use preinstalled toolchains:

```starlark
register_toolchains(
"@rules_foreign_cc//toolchains:preinstalled_pkgconfig_toolchain",
"@rules_foreign_cc//toolchains:preinstalled_make_toolchain",
)
```

and remove build logs:

```diff
diff --git a/foreign_cc/private/framework.bzl b/foreign_cc/private/framework.bzl
index 33129b8..7326107 100644
--- a/foreign_cc/private/framework.bzl
+++ b/foreign_cc/private/framework.bzl
@@ -616,7 +616,7 @@ def wrap_outputs(ctx, lib_name, configure_name, script_text, env_prelude, build_
cleanup_on_success_function = create_function(
ctx,
"cleanup_on_success",
- "rm -rf $$BUILD_TMPDIR$$ $$EXT_BUILD_DEPS$$",
+ "rm -rf $$BUILD_TMPDIR$$ $$EXT_BUILD_DEPS$$ && echo > $$BUILD_LOG$$",
)
cleanup_on_failure_function = create_function(
ctx,
```

Contributor guide

Open the contributing guide

Research direction

Start with foreign_cc/private/framework.bzl, especially the cleanup functions and the generated build logs mentioned in the report. Reproduce the issue with the shown bazel build, bazel clean, and sha256sum commands, then inspect the make and pkg-config toolchain definitions. Done means repeated clean builds produce identical toolchain binaries and do not embed varying sandbox paths in logs.

Written by the indexing model from the issue text.

Assessment

Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.