bazelbuild / bazelbuild/continuous-integration
Gerrit: Git mirror checkout fails for relative submodule URLs
- Dominant language
- Python
- Stars
- 302
- Forks
- 194
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 41
Description
## Summary
The Bazel Buildkite Gerrit pipeline fails during checkout before any
project commands run. The failure happens while the Buildkite git mirror
checkout path tries to mirror Gerrit's submodules.
Gerrit uses valid relative submodule URLs in `.gitmodules`, for example:
```ini
[submodule "modules/java-prettify"]
path = modules/java-prettify
url = ../java-prettify
```
Git resolves that URL relative to the superproject remote:
```text
https://gerrit.googlesource.com/gerrit.git
+ ../java-prettify
= https://gerrit.googlesource.com/java-prettify
```
The Buildkite mirror checkout path appears to use the literal relative URL
from `.gitmodules` while its current directory is `/var/lib/gitmirrors`.
It then tries to clone `../java-prettify` as a local filesystem path:
```text
$ cd /var/lib/gitmirrors
# Cloning a mirror of the repository to "/var/lib/gitmirrors/---java-prettify"
$ git clone --mirror -v --bare -- ../java-prettify /var/lib/gitmirrors/---java-prettify
fatal: repository '../java-prettify' does not exist
```
This makes checkout fail repeatedly even though the submodule URL is valid
and `git submodule update` would resolve it correctly.
## Example failing build
https://buildkite.com/bazel/gerrit/builds/14305/list?sid=01a02bea-ed50-4f60-b9c9-85982683dc66&tab=output
Relevant log excerpt:
```text
# Git submodules detected
$ git submodule sync --recursive
$ cd /var/lib/gitmirrors
# Cloning a mirror of the repository to "/var/lib/gitmirrors/---java-prettify"
$ git clone --mirror -v --bare -- ../java-prettify /var/lib/gitmirrors/---java-prettify
fatal: repository '../java-prettify' does not exist
# Remote Git mirror: outcome=skipped site=none skip_reason=no-url
Warning: Checkout failed! getting/updating mirror dir for submodules: exit status 128
```
The same failure repeats until checkout gives up:
```text
Error: getting/updating mirror dir for submodules: exit status 128
```
## Why this looks like a mirror checkout bug
The Gerrit pipeline is configured with an absolute repository URL:
```text
https://gerrit.googlesource.com/gerrit.git
```
The CI agent config enables git mirrors:
```text
git-mirrors-path="/var/lib/gitmirrors"
git-clone-mirror-flags="-v --bare"
```
The failing mirror path:
```text
/var/lib/gitmirrors/---java-prettify
```
suggests the mirror code sanitized the raw relative URL `../java-prettify`
instead of first resolving it against the superproject remote.
Expected resolved URL:
```text
https://gerrit.googlesource.com/java-prettify
```
Expected mirror behavior:
```text
git clone --mirror -v --bare -- \
https://gerrit.googlesource.com/java-prettify \
/var/lib/gitmirrors/
```
## Expected behavior
When mirroring submodules, relative submodule URLs should be resolved using
Git's normal semantics: resolve them relative to the superproject remote URL,
not relative to the current working directory of the mirror cache.
For Gerrit:
```text
../java-prettify
-> https://gerrit.googlesource.com/java-prettify
../jgit
-> https://gerrit.googlesource.com/jgit
../plugins/replication
-> https://gerrit.googlesource.com/plugins/replication
```
If the mirror code cannot safely resolve a relative submodule URL, it should
skip the mirror optimization for that submodule and let `git submodule update`
perform the normal Git resolution.
## Actual behavior
The mirror checkout path tries to clone the unresolved relative URL from
`/var/lib/gitmirrors`:
```text
git clone --mirror -v --bare -- ../java-prettify \
/var/lib/gitmirrors/---java-prettify
```
That fails because `/var/lib/java-prettify` does not exist.
## Notes
This is not a request to change Gerrit's `.gitmodules` to absolute URLs.
Relative submodule URLs are valid Git behavior and are useful for repositories
that are mirrored or forked.
The fix should be in the checkout/mirror layer:
1. Resolve relative submodule URLs against the parent repository remote URL
before creating/updating the mirror.
2. Base the mirror directory name on the resolved URL, not the raw relative
URL.
3. Fall back to normal `git submodule update` behavior if the URL cannot be
resolved.
Contributor guide
Research direction
Start with the checkout/mirror path that emits the failing git clone command, using build 14305 and the provided log to reproduce the failure. Trace how .gitmodules URLs and the parent remote are handled; done means relative URLs are resolved before mirroring, or safely fall back to normal git submodule update behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, python
- Domain
- ci-cd, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100