False external repository modification warnings/refetches on macOS
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the bug:
Bazel sometimes reports external repository files as externally modified and refetches those repositories on the next invocation, even though I have not intentionally modified anything under Bazel's external repositories.
This looks similar to #29590 / #29701, but the known hardlink/hermetic Linux sandbox explanation does not seem to fit this environment:
- Host is macOS, not Linux.
- I am not using the hermetic Linux sandbox.
- `--experimental_repository_cache_hardlinks` is not enabled.
- The workspace does set `--repo_contents_cache=` because of a git-worktree-related issue.
- The workspace manually sets `--repository_cache=/var/tmp/_bazel_$USER/cache/repos/v1`.
I see this across multiple external repos and in multiple workspaces. @fmeum mentioned that the Linux hardlink case was the only known false-positive source and asked me to file an issue for this macOS case.
Expected behavior: Bazel should not warn/refetch unless the fetched repository contents were actually modified externally.
### Which category does this issue belong to?
_No response_
### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
I do not yet have a minimal public repro for the original spontaneous case. The issue is observed during normal Bazel invocations in a large Bzlmod-based iOS workspace on macOS.
The workspace configuration includes:
- Bzlmod
- macOS host
- no hermetic Linux sandbox
- no `--experimental_repository_cache_hardlinks`
- `--repo_contents_cache=` set
- `--repository_cache=/var/tmp/_bazel_$USER/cache/repos/v1`
Example warnings from one invocation:
```text
WARNING: Repository '@@bazel_skylib+' will be fetched again since the file 'rules/private/bzl_library.bzl' has been modified externally. External modifications can lead to incorrect builds.
WARNING: Repository '@@rules_swift+' will be fetched again since the file 'swift/swift_module_mapping.bzl' has been modified externally. External modifications can lead to incorrect builds.
WARNING: Repository '@@rules_python+' will be fetched again since the file 'python/private/pypi/hub_repository.bzl' has been modified externally. External modifications can lead to incorrect builds.
WARNING: Repository '@@bazel_skylib+' will be fetched again since the file 'lib/modules.bzl' has been modified externally. External modifications can lead to incorrect builds.
WARNING: Repository '@@rules_apple+' will be fetched again since the file 'apple/internal/aspects/app_intents_aspect.bzl' has been modified externally. External modifications can lead to incorrect builds.
WARNING: Repository '@@cgrindel_bazel_starlib+' will be fetched again since the file 'tools/git/git_toolchains.bzl' has been modified externally. External modifications can lead to incorrect builds.
WARNING: Repository '@@protobuf+' will be fetched again since the file 'bazel/private/proto_info.bzl' has been modified externally. External modifications can lead to incorrect builds.
WARNING: Repository '@@rules_swift+' will be fetched again since the file 'swift/internal/swift_autoconfiguration.bzl' has been modified externally. External modifications can lead to incorrect builds.
WARNING: Repository '@@swiftlint+' will be fetched again since the file '.bazelignore' has been modified externally. External modifications can lead to incorrect builds.
WARNING: Repository '@@apple_support+' will be fetched again since the file 'crosstool/BUILD' has been modified externally. External modifications can lead to incorrect builds.
```
Reduction findings:
I tried to reproduce this in a fresh worktree/output base. Repeated `bazel info release`, repeated `bazel mod graph`, and a clean standalone Bzlmod workspace using `bazel_skylib` did not reproduce the warning by themselves.
I was able to reduce the warning mechanism to a ctime-only metadata change on macOS. In a standalone workspace using Bazel `9.1.1rc1`, Bzlmod, `--repository_cache=/var/tmp/_bazel_$USER/cache/repos/v1`, and `--repo_contents_cache=`, a metadata-only change to an external repository file triggers the exact warning even though size, mtime, inode, and contents are unchanged.
Minimal standalone workspace:
```starlark
# MODULE.bazel
module(name = "external_mod_repro")
bazel_dep(name = "bazel_skylib", version = "1.9.0")
```
```starlark
# BUILD.bazel
genrule(
name = "copy_skylib_file",
srcs = ["@bazel_skylib//:LICENSE"],
outs = ["copied_bzl_library.bzl"],
cmd = "cp $(location @bazel_skylib//:LICENSE) $@",
)
```
Commands:
```sh
bazel \
--output_base=/tmp/bazel-external-mod-repro-output \
build \
--repository_cache=/var/tmp/_bazel_$USER/cache/repos/v1 \
--repo_contents_cache= \
--enable_bzlmod \
--lockfile_mode=off \
//:copy_skylib_file
file_path=/tmp/bazel-external-mod-repro-output/external/bazel_skylib+/LICENSE
stat -f 'before | links=%l | size=%z | mtime=%m | ctime=%c | inode=%i' "$file_path"
xattr -w com.reddit.repro bazel-external-mod-repro "$file_path"
stat -f 'after | links=%l | size=%z | mtime=%m | ctime=%c | inode=%i' "$file_path"
bazel \
--output_base=/tmp/bazel-external-mod-repro-output \
build \
--repository_cache=/var/tmp/_bazel_$USER/cache/repos/v1 \
--repo_contents_cache= \
--enable_bzlmod \
--lockfile_mode=off \
//:copy_skylib_file
```
Output from the metadata-only change:
```text
before | links=1 | size=11358 | mtime=946684800 | ctime=1781037904 | inode=964892192
after | links=1 | size=11358 | mtime=946684800 | ctime=1781037944 | inode=964892192
WARNING: Repository '@@bazel_skylib+' will be fetched again since the file 'LICENSE' has been modified externally. External modifications can lead to incorrect builds.
```
Creating a hardlink to the same external repo file on macOS also triggers the warning:
```sh
file_path=/tmp/bazel-external-mod-repro-output/external/bazel_skylib+/LICENSE
rm -f /tmp/bazel-external-mod-repro-hardlink
stat -f 'before | links=%l | size=%z | mtime=%m | ctime=%c | inode=%i' "$file_path"
ln "$file_path" /tmp/bazel-external-mod-repro-hardlink
stat -f 'after | links=%l | size=%z | mtime=%m | ctime=%c | inode=%i' "$file_path"
```
```text
before | links=1 | size=11358 | mtime=946684800 | ctime=1781037944 | inode=964907479
after | links=2 | size=11358 | mtime=946684800 | ctime=1781037959 | inode=964907479
WARNING: Repository '@@bazel_skylib+' will be fetched again since the file 'LICENSE' has been modified externally. External modifications can lead to incorrect builds.
```
So the current status is: I do not yet know what process is causing the ctime-only changes in the original workspace, but on macOS a ctime-only metadata change is sufficient to produce the exact warning/refetch behavior without `--experimental_repository_cache_hardlinks`.
### Which operating system are you running Bazel on?
macOS 26.6 (25G5028f)
### What is the output of `bazel info release`?
release 9.1.1rc1
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
N/A
### What's the output of `git remote get-url origin; git rev-parse HEAD` ?
```text
Private workspace; remote URL and exact commit can be provided privately if needed.
```
### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.
Unknown. I have not bisected yet because I do not have a minimal public repro for the original spontaneous case.
### Have you found anything relevant by searching the web?
Related issue/PR: #29590 / #29701 cover false positives from hardlinks with the hermetic Linux sandbox. The reduction above shows the same warning can be triggered by ctime-only changes on macOS as well, without repository cache hardlinks enabled.
### Any other information, logs, or outputs that you want to share?
This was observed with Bazel `release 9.1.1rc1` on macOS 26.6 (25G5028f). The affected workspace is private, so I did not include its remote URL in this public issue draft.
Contributor guide
Research direction
Start with the standalone MODULE.bazel and BUILD.bazel reproduction, using the provided Bazel build commands and metadata-only xattr and hardlink changes under the external repository. Trace the external repository validation that treats unchanged contents as modified on macOS. Done means ctime-only metadata changes no longer trigger warnings or refetches, while genuine external modifications still do.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, macos
- Domain
- build-system, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100