Allow git_override to specify pathspec for --recurse_submodules
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the feature request:
**The feature request**
We request to add an attribute to `git_override`:
```python
git_override(
module_name = "kleaf",
remote = "https://android.googlesource.com/kernel/superproject",
commit = "1c03b74f03bb7a45856b8e131a857c238af12033",
init_submodules = True,
recurse_submodules_pathspec = ":(attr:ddk)"
)
```
... or any solution that can allow us to pull only submodules with the given attribute to achieve the behavior of `git clone --recurse-submodules=':(attr:ddk)' ...`
### Which category does this issue belong to?
External Dependency
### What underlying problem are you trying to solve with this feature?
**Background**
We (the [Kleaf](https://android.googlesource.com/kernel/build/+/refs/heads/main/kleaf/README.md) team) are working on a feature that needs to pull a subtree of a source tree as a module that builds a build number on ci.android.com. The way we design it is the following:
- From ci.android.com, for a given build number, infer the superproject SHA of the [kernel superproject](https://android.googlesource.com/kernel/superproject)
- Add the following to MODULE.bazel:
```python
git_override(
module_name = "kleaf",
remote = "https://android.googlesource.com/kernel/superproject",
commit = "1c03b74f03bb7a45856b8e131a857c238af12033",
init_submodules = True,
)
```
However, this will pull the whole superproject. We would like to be able to pull only projects with certain .gitattributes ([example](https://android.googlesource.com/kernel/superproject/+/refs/heads/common-android-mainline/.gitattributes). These git attributes were originally from the `groups` of a repo manifest ([example](https://android.googlesource.com/kernel/manifest/+/refs/heads/common-android-mainline/default.xml#8))
So, to pull groups=ddk projects, with `repo` we would do
```shell
repo init --groups=ddk ...
```
With Git submodules we would do
```shell
git clone --recurse-submodules=':(attr:ddk)' ...
```
However, there is no way to specify `--recurse-submodules=` in `git_override`.
**Alternatives**
One may suggest us to use `use_repo_rule` and implement our own repo_rule instead. However, I need a full module so that bazel will read its `MODULE.bazel` and resolve the dependencies, not just a single repository.
We also come up with some hacky alternative solutions, example:
```python
git_override(
module_name = "kleaf",
remote = "https://android.googlesource.com/kernel/superproject",
commit = "1c03b74f03bb7a45856b8e131a857c238af12033",
patch_cmds = [
"tools/filter_submodules.sh -g ddk",
],
)
```
where the `tools/filter_submodules.sh` contains:
```sh
GROUP=ddk #
git rm -r $(git check-attr $GROUP -- $(git submodule status | cut -f2 -d' ') | \
grep -v "$GROUP: set" | cut -f1 -d':')
git commit -am'Remove unrelated projects'
git submodule update --init --recursive
```
### Which operating system are you running Bazel on?
Linux
### What is the output of `bazel info release`?
release 7.0.2
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
_No response_
### What's the output of `git remote get-url origin; git rev-parse HEAD` ?
_No response_
### Have you found anything relevant by searching the web?
Git pathspecs: https://css-tricks.com/git-pathspecs-and-how-to-use-them/#aa-attr
Git submodules: https://git-scm.com/docs/git-clone#Documentation/git-clone.txt---recurse-submodulesltpathspecgt
### Any other information, logs, or outputs that you want to share?
_No response_
Contributor guide
Research direction
Start at the git_override entry point and its existing init_submodules handling; compare the submodule invocation with Git's documented --recurse-submodules= behavior. Done means git_override can select submodules by pathspec while preserving module loading, with coverage for the requested usage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100