bazelbuild / bazelbuild/bazel

Make bazel-bin more reliable

Open
#15,005 5 comments 0 reactions 0 assignees View on GitHub
P2 team-Configurability type: feature request
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the problem / feature request:

Targets don't appear in `bazel-bin` in mixed-configuration builds. See, for example, https://github.com/bazelbuild/bazel/issues/12389.

`--use_top_level_targets_for_symlinks` partially solves this problem, but [not completely](https://github.com/bazelbuild/bazel/issues/12389#issuecomment-1062298182).

### Any other information, logs, or outputs that you want to share?

I explored this and wrote some prototype code.

Conclusion: there's no trivial way to support multi-config `bazel-bin`, if we consider "trivial" = "super-low chance of depot breakages"

The code I wrote added a new postprocessing step: if an output isn't in the top-level configuration, add a new symlink under `bazel-bin` to it. i.e.:

bazel-bin/mydir/myapp -> bazel-out/x86-fastbuild/bin/mydir/myapp

This roughly works but sacrifices Bazel's correctness guarantees:

1. What if a legitimate file already exists at `bazel-bin/mydir/myapp`?
1. We could adapt the logic to "only create the symlink when `bazel-bin/mydir/myapp` doesn't exist
1. But what if `bazel-bin/mydir/myapp` is the symlink from the last build and we need to update it?
1. We could adapt the logic to "delete existing `bazel-bin/mydir/myapp`" if it's a symlink
1. What if the targets legitimately output symlinks?
1. We could adapt the logic to check if the symlink reaches out to a different `bazel-out` root
1. But target outputs can reach out to different roots (like if they inherit outputs from their children)
1. If we accidentally preserve an old symlink we should have deleted, it may point to the wrong value for the current build
1. So it's an unresolvable mess trying to preserve correctness.

Since by definition this problem occurs when mixing configs, there is no perfect solution in which `bazel-bin` points to a single config base.

Therefore, to stay correct we'd need to refactor `bazel-bin` to mix configs. This is fine: `bazel-bin -> bazel-out/mixedconfigs/bin` and then symlink forest all target outputs. And clean out `bazel-out/mixedconfigs/` for each build.

But which outputs do we symlink from mixedconfigs? Just the targets? Also its runfiles? Do debuggers need debug symbols? We can no longer take for granted that every output in `bazel-bin/` can access all its peer outputs in the same directory. Only if we symlink them all.

As a worst-case backup we could symlink the total union of all `bazel-out/...` paths, and give up if there are any conflicts. This seems like overkill though.

The ultimate answer depends on how people really depend on `bazel-bin` in practice.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the mixed-configuration behavior described here and the linked issue #12389, including the limits of --use_top_level_targets_for_symlinks. Then evaluate the proposed mixedconfigs symlink-forest approach against the listed correctness concerns and determine which outputs, runfiles, and debug symbols users actually depend on.

Written by the indexing model from the issue text.

Assessment

Domain
build-system
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.