bazel-contrib / bazel-contrib/rules_python

Avoid downloading protobuf dependancy when not needed

Aperta
#2,543 5 commenti 0 reazioni 0 assegnatari Vedi su GitHub
help wanted need: upstream support
Lingua principale
Starlark
Stelle
688
Fork
721
Merge medio
15h 7m
PR unite (30g)
76

Descrizione

Hi we recently migrated a project from workspace to bzlmod and noticed our builds now pull in more things from the internet as a result.
In particular we saw that our builds started to download protobuf (from https://github.com/protocolbuffers/protobuf/releases/download/v29.0-rc2/protobuf-29.0-rc2.zip).
This was despite none of our targets referencing (either directly nor indirectly) `py_proto_library` (or any other protobuf target from `rules_python` or any other module).

It seems that in a workspace project dependencies are fetched lazily [1] whereas in a bzlmod project, dependencies are fetched eagerly.

Reading the source of this library, I saw a comment in the MODULE.bzl https://github.com/bazelbuild/rules_python/blob/29fdcc2c292d64e7c66746d1c701ab67aa05c79b/MODULE.bazel#L12 that suggests the `protobuf` dep will be loaded only if the `py_proto_library` is used. However, that doesn't match my tests where `protobuf` is download by all projects that depend on `rules_python` even if they do not use _any_ rules from it [2].

I am wondering

1. if there is a way to use `rules_python` without pulling `protobuf` if we do not need protobuf support?
2. If the comment linked above needs updating/removing?

#### Related

I think https://github.com/bazelbuild/rules_python/issues/2498 is similar although that is about rules_kotlin and is more about toolchains fetched as module extensions rather than modules downloaded directly.

#### Notes

1: https://bazel.build/extending/repo#when_is_the_implementation_function_executed
2: You can test this by creating a custom registry (https://bazel.build/external/registry) and modifying the `source.json` file of all the protobuf modules to set the `url` field to something that does not exist. Building a bzlmod project that uses rules_python but does not have any `py_proto_library` targets will then fail showing that bazel does try to fetch protobuf.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start by reading MODULE.bzl around line 12 and tracing how the protobuf dependency relates to py_proto_library under bzlmod. Reproduce the behavior with a custom registry whose protobuf source URLs are invalid, using a project that depends on rules_python but defines no protobuf targets. Done means the project no longer attempts to fetch protobuf unless the relevant support is used, with the comment or documentation matching the behavior.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
build-system
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.