bazelbuild / bazelbuild/bazel

Repository caching bug with implicit glob watching

Open
#28,980 1 comment 0 reactions 0 assignees View on GitHub
help wanted P2 team-ExternalDeps type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

There appears to be a bug where globs and the implicit watches they register in a repository will always trigger repository invalidation. It seems related to symlinks and change detection of the files implementating a repository.

Given a repo rule that looks roughly like this:

```
def _impl(rctx):
rctx.file("BUILD", "glob('foo/**')")
rctx.file("cache/marker", "")
rctx.symlink("cache/", "foo/cache")
print("hit")
```

Expected behavior:
```
bazel clean
bazel build # prints hit
touch /cache/x.txt
bazel build # doesn't print hit

bazel build # prints hit
touch /cache/x.txt
bazel build # doesn't print hit
```

Actual behavior:

```
bazel clean
bazel build # prints hit
touch /cache/x.txt
bazel build # doesn't print hit

bazel build # prints hit
touch /cache/x.txt
bazel build # prints hit
```

Note how in expected, it properly caching the repo after 1 invocation. In the actual, it has ceased caching entirely. In the actual output, when it warms the repo is re-running due to external modification, it prints a message like `warning: external//cache/x.txt changed`. This is feels notable because _usually_ the file path it prints doesn't include `external/`. Gives me "something is confused by a symlink" vibes.

Why is the "touch file in repo" step there? That may seem a bit strange, but it how Python behaves. Given `lib/python3.11/foo.py`, it creates `lib/python3.11/__pycache__/foo.pyc`. In the actual code I ran into this bug with, the symlink() call is linking e.g. `lib/python3.11/__pycache__ -> unwatched_dir/lib/python3.11/__pycache__`, and there is a `glob(lib/**)`

This seems to _only_ happen with the combination of a glob, symlinks, and modifying the bzl file implementing a repo rule.

### Which category does this issue belong to?

_No response_

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Use this commit: https://github.com/rickeylev/rules_python/commit/9c077f2fbff1143f0b3a6d6dd7696e67f06e95d5

And run `tests/repro/repro.sh`. Modify the `python/private/python_repository.bzl` file, then run repro.sh again. On the second and subsequent runs, the repo will refuse the stay cached
```
git clone https://github.com/rickeylev/rules_python/
git checkout 9c077f2fbff1143f0b3a6d6dd7696e67f06e95d5
tests/repro/repro.sh

tests/repro/repro.sh
```

### Which operating system are you running Bazel on?

linux

### What is the output of `bazel info release`?

9.0

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?

```text

```

### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

_No response_

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Research direction

Check out commit 9c077f2fbff1143f0b3a6d6dd7696e67f06e95d5 and run tests/repro/repro.sh to reproduce the cache invalidation. Inspect python/private/python_repository.bzl alongside the glob and symlink behavior described in the issue. Done means modifying the repository rule still invalidates once, while later changes under the linked cache directory do not rerun it.

Written by the indexing model from the issue text.

Assessment

Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.