`git_repository` checkout is non-hermetic w.r.t. host `core.autocrlf`/`core.eol`
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 75
Description
### Description of the bug:
`@bazel_tools//tools/build_defs/repo:git.bzl`'s `git_repository` / `new_git_repository` shell out to the system `git`, inheriting the host's ambient Git configuration.
The checked-out working tree is therefore not reproducible across machines: `core.autocrlf` and `core.eol` rewrite line endings on checkout.
The common case is Git for Windows, whose installer default `core.autocrlf=true` rewrites `LF` to `CRLF` on checkout.
So a repository that stores `LF` is materialized with `CRLF` on a Windows developer's or CI's machine, but `LF` elsewhere: the bytes a rule sees depend on who fetched them.
This breaks reproducibility (e.g. byte-sensitive rules, like code generators), and pushes to abandon `git_repository` for `http_archive`.
This is the line-ending facet of the broader non-hermeticity reported in bazelbuild/bazel#10909 ("git_worker.bzl appears non-hermetic", closed as not planned): `git_worker.bzl`'s `_execute` runs `git` with the host environment and the host Git config.
### Which category does this issue belong to?
External Dependency
### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
1. Create a local Git repo, commit one `LF`-terminated text file, and tag it:
```
git init origin && cd origin
printf 'alpha\nbeta\n' > data.txt
git -c core.autocrlf=false add -A && git -c core.autocrlf=false commit -m init
git tag first
```
2. Reference it from `MODULE.bazel`:
```
git_repository = use_repo_rule("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")
git_repository(name = "ext", remote = "file:///.../origin/.git", tag = "first",
build_file_content = 'filegroup(name = "data", srcs = ["data.txt"])')
```
3. Give the fetch the Git-for-Windows default config (reproducible on any OS):
`--repo_env=HOME=`
(or simply `git config --global core.autocrlf true`).
4. `bazel build @ext//:data` and inspect the checked-out `data.txt` in the external repo.
Expected: `data.txt` is checked out byte-for-byte as committed (`LF`).
Actual: `data.txt` is checked out with `CRLF`; the same build with `core.autocrlf=false` yields `LF`.
The checkout depends on the host Git config.
`core.eol=crlf` reaches the same outcome for any file Git treats as text (e.g. via a global/system `gitattributes` marking files `text`).
### Which operating system are you running Bazel on?
Linux, macOS, Windows
### What is the output of `bazel info release`?
release 9.1.1
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
_No response_
### What's the output of `git remote get-url origin; git rev-parse HEAD` ?
```text
git@github.com:rdesgroppes/bazel.git
d424170a51cae245f9828f000db45460301269ba
```
### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.
_No response_
### Have you found anything relevant by searching the web?
- #10909, closed as not planned: the general form of this problem.
Git for Windows defaults to `core.autocrlf=true`, which rewrites LF to CRLF on checkout:
- https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration
> If you're on a Windows machine, set it to `true` — this converts LF endings into CRLF when you check out code:
- https://docs.github.com/en/get-started/git-basics/configuring-git-to-handle-line-endings
> Configure Git to ensure line endings in files you checkout are correct for Windows. For compatibility, line endings are converted to Unix style when you commit files.
- https://dev.to/kevinshu/git-and-normalization-of-line-endings-228j
> This is the default value pushed by the installer on Windows systems
### Any other information, logs, or outputs that you want to share?
A rather trivial fix would be to pin the line-ending knobs per invocation in `git_worker.bzl`'s `_execute`:
```diff
- start = ["git", "-c", "core.fsmonitor=false"]
+ start = ["git", "-c", "core.fsmonitor=false", "-c", "core.autocrlf=false", "-c", "core.eol=lf"]
```
`-c name=value` overrides every config source (system / global / XDG / local) at the highest precedence and works on Git >= 1.7.2, so it also covers users on older Git, unlike `GIT_CONFIG_GLOBAL` / `GIT_CONFIG_SYSTEM` (git >= 2.32).
Properties:
- an in-tree `.gitattributes` still wins over `core.eol`, so a repo's own line-ending policy (e.g. `eol=crlf`) is preserved,
- the checkout reproduces the committed bytes: `LF` stays `LF`, and a `CRLF` blob stays `CRLF` (the checkout/smudge path only ever adds `CR`, never strips it: stripping happens only on check-in, which `git_repository` never does).
Scope: this is narrow to line endings.
Other content-altering ambient knobs (`core.symlinks`, clean/smudge filters such as Git LFS) and the env-var non-hermeticity that #10909 centers on are deliberately out of scope, to avoid stripping config the fetch legitimately needs (proxies, credential helpers, `url.insteadOf`).
Contributor guide
Research direction
Read @bazel_tools//tools/build_defs/repo:git.bzl and git_worker.bzl, starting at _execute where the git command is assembled. Reproduce the issue with the provided local repository and a core.autocrlf=true configuration, then verify that git_repository and new_git_repository produce committed line endings regardless of that host setting while preserving the repository’s own .gitattributes policy.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100