bazelbuild / bazelbuild/bazel

`git_repository` checkout is non-hermetic w.r.t. host `core.autocrlf`/`core.eol`

Open
#30,026 0 comments 0 reactions 0 assignees View on GitHub
area-Bzlmod area-EngProd area-Windows team-Core type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 18h
Merged PRs (30d)
75

Description

### Description of the bug:
`@bazel_tools//tools/build_defs/repo:git.bzl`'s `git_repository` / `new_git_repository` shell out to the system `git`, inheriting the host's ambient Git configuration.
The checked-out working tree is therefore not reproducible across machines: `core.autocrlf` and `core.eol` rewrite line endings on checkout.

The common case is Git for Windows, whose installer default `core.autocrlf=true` rewrites `LF` to `CRLF` on checkout.
So a repository that stores `LF` is materialized with `CRLF` on a Windows developer's or CI's machine, but `LF` elsewhere: the bytes a rule sees depend on who fetched them.
This breaks reproducibility (e.g. byte-sensitive rules, like code generators), and pushes to abandon `git_repository` for `http_archive`.

This is the line-ending facet of the broader non-hermeticity reported in bazelbuild/bazel#10909 ("git_worker.bzl appears non-hermetic", closed as not planned): `git_worker.bzl`'s `_execute` runs `git` with the host environment and the host Git config.

### Which category does this issue belong to?
External Dependency

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
1. Create a local Git repo, commit one `LF`-terminated text file, and tag it:
```
git init origin && cd origin
printf 'alpha\nbeta\n' > data.txt
git -c core.autocrlf=false add -A && git -c core.autocrlf=false commit -m init
git tag first
```
2. Reference it from `MODULE.bazel`:
```
git_repository = use_repo_rule("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
git_repository(name = "ext", remote = "file:///.../origin/.git", tag = "first",
build_file_content = 'filegroup(name = "data", srcs = ["data.txt"])')
```
3. Give the fetch the Git-for-Windows default config (reproducible on any OS):
`--repo_env=HOME=`
(or simply `git config --global core.autocrlf true`).
4. `bazel build @ext//:data` and inspect the checked-out `data.txt` in the external repo.

Expected: `data.txt` is checked out byte-for-byte as committed (`LF`).
Actual: `data.txt` is checked out with `CRLF`; the same build with `core.autocrlf=false` yields `LF`.
The checkout depends on the host Git config.

`core.eol=crlf` reaches the same outcome for any file Git treats as text (e.g. via a global/system `gitattributes` marking files `text`).

### Which operating system are you running Bazel on?
Linux, macOS, Windows

### What is the output of `bazel info release`?
release 9.1.1

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
_No response_

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?
```text
git@github.com:rdesgroppes/bazel.git
d424170a51cae245f9828f000db45460301269ba
```

### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.
_No response_

### Have you found anything relevant by searching the web?
- #10909, closed as not planned: the general form of this problem.

Git for Windows defaults to `core.autocrlf=true`, which rewrites LF to CRLF on checkout:
- https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration
> If you're on a Windows machine, set it to `true` — this converts LF endings into CRLF when you check out code:
- https://docs.github.com/en/get-started/git-basics/configuring-git-to-handle-line-endings
> Configure Git to ensure line endings in files you checkout are correct for Windows. For compatibility, line endings are converted to Unix style when you commit files.
- https://dev.to/kevinshu/git-and-normalization-of-line-endings-228j
> This is the default value pushed by the installer on Windows systems

### Any other information, logs, or outputs that you want to share?
A rather trivial fix would be to pin the line-ending knobs per invocation in `git_worker.bzl`'s `_execute`:
```diff
- start = ["git", "-c", "core.fsmonitor=false"]
+ start = ["git", "-c", "core.fsmonitor=false", "-c", "core.autocrlf=false", "-c", "core.eol=lf"]
```

`-c name=value` overrides every config source (system / global / XDG / local) at the highest precedence and works on Git >= 1.7.2, so it also covers users on older Git, unlike `GIT_CONFIG_GLOBAL` / `GIT_CONFIG_SYSTEM` (git >= 2.32).
Properties:
- an in-tree `.gitattributes` still wins over `core.eol`, so a repo's own line-ending policy (e.g. `eol=crlf`) is preserved,
- the checkout reproduces the committed bytes: `LF` stays `LF`, and a `CRLF` blob stays `CRLF` (the checkout/smudge path only ever adds `CR`, never strips it: stripping happens only on check-in, which `git_repository` never does).

Scope: this is narrow to line endings.
Other content-altering ambient knobs (`core.symlinks`, clean/smudge filters such as Git LFS) and the env-var non-hermeticity that #10909 centers on are deliberately out of scope, to avoid stripping config the fetch legitimately needs (proxies, credential helpers, `url.insteadOf`).

Contributor guide

Open the contributing guide

Research direction

Read @bazel_tools//tools/build_defs/repo:git.bzl and git_worker.bzl, starting at _execute where the git command is assembled. Reproduce the issue with the provided local repository and a core.autocrlf=true configuration, then verify that git_repository and new_git_repository produce committed line endings regardless of that host setting while preserving the repository’s own .gitattributes policy.

Written by the indexing model from the issue text.

Assessment

Tech stack
git
Domain
build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.