bazelbuild / bazelbuild/bazel

FR: Store resolved repository attributes in the Bzlmod lockfile

Open
#19,026 8 comments 1 reaction 0 assignees View on GitHub
area-Bzlmod P2 team-ExternalDeps type: feature request
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 18h
Merged PRs (30d)
75

Description

### Description of the feature request:

The information about resolved attributes returned by repo rules and collected by `--experimental_repository_resolved_file` should be stored in the Bzlmod lockfile.

Repository rules already use this information to lazily (that is, when evaluated) return (often cryptographic) identifiers that make their results fully reproducible, which can also result in faster fetches (via Bazel's repository cache).

If `MODULE.bazel.lock` could be updated with this information incrementally, users would have strong "trust on first use" guarantees without having to manually provide cryptographic identifiers in an extension-dependent manner.

### What underlying problem are you trying to solve with this feature?

Lockfiles for package managers usually serve two purposes:
1. Pinning a particular resolution of all (transitive) dependency requirements to a set of deps and their versions.
2. Recording cryptographic identifiers for the artifacts representing these resolved deps, e.g. hashes of archives or their contents.

With the Bzlmod lockfile available with Bazel 6.3.0, there is a general solution for 1. that module extensions can use instead of rolling their own one.

However, 2. remains unsolved: When wrapping an external package manager for the fictitious `foo` language into a module extension `foo_deps`, in order to benefit from Bazel's repository cache, users either have to manually specify hashes of the resolved deps in their `MODULE.bazel` file or a file referenced from it (such as `go.sum` for Go) or rely on custom module extension/repo rule logic to generate such a file (such as `rules_jvm_external`'s concept of pinning). The latter usually requires eagerly fetching all repos once to generate the lockfile, which can be slow. Since this process differs between module extensions, it results in additional friction beyond just updating Starlark files.

### Which operating system are you running Bazel on?

N/A

### What is the output of `bazel info release`?

N/A

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

_No response_

### Have you found anything relevant by searching the web?

All module extensions for languages that do not come with a well-established lockfile format currently need to come up with their own format. This includes existing rulesets such as `rules_jvm_external` as well as new developments (e.g. [for PHP](https://bazelbuild.slack.com/archives/C014RARENH0/p1684506698597549).

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by tracing how --experimental_repository_resolved_file collects resolved repository attributes and how MODULE.bazel.lock is read and updated. Compare the existing module-extension lockfile flow with repository-rule results. Done means the lockfile records these attributes incrementally and supports reproducible, cacheable repository resolution.

Written by the indexing model from the issue text.

Assessment

Domain
build-system
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.