bazelbuild / bazelbuild/bazel

Hermetic sandbox reports cached coproduced runfile symlink modified during execution after sequence of repo refetches

Open
#29,832 0 comments 0 reactions 0 assignees View on GitHub
team-Core type: bug untriaged
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

When there is:
* experimental hermetic linux sandbox used;
* a target generating multiple files, defined by the repository rule, with some output files' content varying and some stable (with respect to repo refetches);
* a runfile symlink (`root_symlinks` or `symlinks`) is present in the runfiles along with the stable file it targets;
* varying file is also present in the runfiles;
* these runfiles are propagated via intermediary tool, which is an executable;
* a remote cache (e.g. disk cache) involved;
* a specific sequence of repository rule refetches in subsequent `bazel build` invocations;

then the action needing that tool may fail with:
```
I/O exception during sandboxed execution: input dependency … was modified during execution.
```

Reproducing this is quite reliable (doesn't seem to be a race), but requires a specific sequence of `bazel` invocations.

### Which category does this issue belong to?

Core

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

See the minimal example: https://github.com/aszady/bazel-playground/tree/external-cogen-stale-symlink

Runs against 4 different Bazel versions: https://github.com/aszady/bazel-playground/actions/runs/27430561587/job/81079501958

### Which operating system are you running Bazel on?

ubuntu-latest

### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

Regression after https://github.com/bazelbuild/bazel/commit/50e228bcada3e10f5c7ada72e7da0a5fa780febe

Contributor guide

Open the contributing guide

Research direction

Start with the minimal example in the external cogen-stale-symlink repository and reproduce the reported sequence of Bazel build invocations on the experimental hermetic Linux sandbox. Compare behavior across the four Bazel versions and investigate the regression after commit 50e228bcada3e10f5c7ada72e7da0a5fa780febe. Done means the cached coproduced runfile symlink no longer triggers a false modified-input error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, linux
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.