Make bazel-bin more reliable
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the problem / feature request:
Targets don't appear in `bazel-bin` in mixed-configuration builds. See, for example, https://github.com/bazelbuild/bazel/issues/12389.
`--use_top_level_targets_for_symlinks` partially solves this problem, but [not completely](https://github.com/bazelbuild/bazel/issues/12389#issuecomment-1062298182).
### Any other information, logs, or outputs that you want to share?
I explored this and wrote some prototype code.
Conclusion: there's no trivial way to support multi-config `bazel-bin`, if we consider "trivial" = "super-low chance of depot breakages"
The code I wrote added a new postprocessing step: if an output isn't in the top-level configuration, add a new symlink under `bazel-bin` to it. i.e.:
bazel-bin/mydir/myapp -> bazel-out/x86-fastbuild/bin/mydir/myapp
This roughly works but sacrifices Bazel's correctness guarantees:
1. What if a legitimate file already exists at `bazel-bin/mydir/myapp`?
1. We could adapt the logic to "only create the symlink when `bazel-bin/mydir/myapp` doesn't exist
1. But what if `bazel-bin/mydir/myapp` is the symlink from the last build and we need to update it?
1. We could adapt the logic to "delete existing `bazel-bin/mydir/myapp`" if it's a symlink
1. What if the targets legitimately output symlinks?
1. We could adapt the logic to check if the symlink reaches out to a different `bazel-out` root
1. But target outputs can reach out to different roots (like if they inherit outputs from their children)
1. If we accidentally preserve an old symlink we should have deleted, it may point to the wrong value for the current build
1. So it's an unresolvable mess trying to preserve correctness.
Since by definition this problem occurs when mixing configs, there is no perfect solution in which `bazel-bin` points to a single config base.
Therefore, to stay correct we'd need to refactor `bazel-bin` to mix configs. This is fine: `bazel-bin -> bazel-out/mixedconfigs/bin` and then symlink forest all target outputs. And clean out `bazel-out/mixedconfigs/` for each build.
But which outputs do we symlink from mixedconfigs? Just the targets? Also its runfiles? Do debuggers need debug symbols? We can no longer take for granted that every output in `bazel-bin/` can access all its peer outputs in the same directory. Only if we symlink them all.
As a worst-case backup we could symlink the total union of all `bazel-out/...` paths, and give up if there are any conflicts. This seems like overkill though.
The ultimate answer depends on how people really depend on `bazel-bin` in practice.
Contributor guide
Research direction
Start by reviewing the mixed-configuration behavior described here and the linked issue #12389, including the limits of --use_top_level_targets_for_symlinks. Then evaluate the proposed mixedconfigs symlink-forest approach against the listed correctness concerns and determine which outputs, runfiles, and debug symbols users actually depend on.
Written by the indexing model from the issue text.
Assessment
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100