bazelbuild / bazelbuild/bazel

Symlinks inside TreeArtifacts may be broken by Bazel and become dangling

Open
#27,068 2 comments 0 reactions 0 assignees View on GitHub
team-Rules-Python type: bug untriaged
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

Discussed with @fmeum in slack.

Encountered while developing https://github.com/aspect-build/rules_py/pull/644.

On Linux, an action which produces a TreeArtifact containing symlinks to input files of the action may (will) subsequently be broken by sandbox cleanup. Under the Linux sandboxing strategy it appears files are presented to actions as a forest of symlinks to their "real" output locations. The canonical (unresolved) path of these files is a action-specific sandbox ala `${output_base}/sandbox/${step}/...` which Bazel may remove at its discretion.

If the user were to say `ln -s $(location :input) $(location :treeartifact)/foo`, naively the resulting link would be one which points to the sandbox-step specific "file" which itself happens to be a link. The resulting TreeArtifact validates _before the sandbox is cleaned up_, but is _invalid thereafter_. This produces spooky build failures where TreeArtifacts containing notionally valid symlinks later become broken.

Setting the `--sandbox_debug` flag persists all step directories and thus "fixes" the bug, which made detailing this behavior a joy.

Example build output tree containing broken symlinks to a step-specific sandbox.

Image

Resolving the input's real location before creating the link "solves" this problem, but is incompatible with remote execution as the resulting links in the TreeArtifact point to absolute host-specific paths.

With sufficient care it _may_ be possible to construct relative symlinks which would be portable, but that seems like a big lift for a tool populating a directory which should be able to be Bazel-agnostic. I suspect that in general it is not possible to do so anyway due to how the tree may be reorganized.

### Which category does this issue belong to?

Core

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

_No response_

### Which operating system are you running Bazel on?

This is Linux specific, behavior does not replicate on MacOS

### What is the output of `bazel info release`?

Bazel 7.3.0 from the official release channel

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?

```text

```

### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

_No response_

### Have you found anything relevant by searching the web?

Potentially related to #22504.

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Linux-only behavior with Bazel 7.3.0 and a TreeArtifact containing symlinks to action inputs. Inspect the sandboxing and TreeArtifact handling paths implicated by the report. Done means links remain valid after sandbox cleanup without relying on --sandbox_debug, while preserving compatibility with remote execution.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.