Capability like Rust Cargo Features in bzlmod
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the feature request:
# Pain point explained
Protobuf has a large single repo to support every language. In WORKSPACE this worked out fine because the users could pick which transitive deps to pull in and choose only the cases that they actually need without paying costs of the rest. However, it came with all the known downsides of workspace that bzlmod is solving.
However with bzlmod its not really possible to have a large library with many deps and have customers only pay the cost of the ones that they care about. Notably this has led to Protobuf holding back Rust support, because having Rust as a dep for all of users is a large dep cost: https://github.com/protocolbuffers/protobuf/issues/23866
Tentatively the solution is supposed to be that Protobuf would move to small language-specific repos, and leverage the "nodep" capability (https://github.com/bazelbuild/bazel/issues/25214) to have the megarepo as a shim for backwards compatibility.
However, I think this case comes with a number of downsides, not least the switching costs for Protobuf team, the fact that all users who stay on the mega-repo will pay the sum of costs of transitive deps or have to migrate their own codebase, but also the final state post-migration actually doesn't seem obviously "good" for Protobuf's desired semantics.
One concrete example of why the model is not a fit for just lang-specific libraries is that we have a pattern of implementing Protobuf with same API but multiple implementations, which are typically other Protobuf libraries. For example, in Python that can be pure-Python, C++Proto, or upb. In Rust it can be backed by C++Proto or upb. In Ruby it can be backed by JavaProto (JRuby), upb, etc.
In the big picture, it may end up being desirable that RubyProto wants to dep onto JavaProto only for some usecases, JavaProto to dep onto C++Proto only for some usecases, and C++Proto depping onto RustProto only for some usecases.
Today there is a single `rust_proto_library()` and only if a global toggle is set will it use C++Proto under the hood, which in bazel the behavior can be controlled by a single global define which swaps out the underlying implementation. Ideally, RustProto should depend onto C++Proto only in the case that people intend to use Rust-on-C++Proto and only depend on upb when no one did ask for that capability.
When users are doing Rust-Cpp interop usecase, I think its not quite the intended model of either "C++Proto deps onto RustProto" or "RustProto deps onto C++Proto", instead it should be the case that users can either use either or both, but semantically there's really just one library that supports both of them (and people using C++-only shouldn't need to pay any costs if they don't adopt Rust).
# Possible Solution
Rust's Crate Deps design has a solution that is a perfect fit for this scenario called "Features". The way it works is that each library can define a set of optional features, any incoming dep can declare that they want some optional features to be on. The final resolution will have the union of all features that anyone has enabled (an outgoing dep specification cannot demand a feature be disabled, only enabled). Then the library can have both conditional-compilation against those features, as well as conditional-deps from it. So features must be strictly additive behavior and all-features-on must always be legal (no "you can turn on X or Y but not both").
This also will allow for a single module to (for example) have cleanly support for both Rust and C++ even if those two implementations are mutually tied together when both are enabled. Such a library could expose _no_ behavior at all by default, and have two features: "EnableRust" and "EnableCpp". All incoming deps would choose 1 or both, and the final resolution will be the union of them (which may be both even if no incoming dep asks for both when e.g. one dep asks for Rust and a second dep asks for Cpp).
And then a set of targets / starlark rules / etc could be guarded by "these targets/rules only exist if the corresponding feature has been enabled", and the outgoing bzlmod deps are similarly tagged with the corresponding features and so are only included if the feature is requested.
I don't believe this is the same thing as Optional Dependencies as described as https://github.com/bazelbuild/bazel/issues/25214; really the incoming deps should be able to specify an abstracted _feature set_ that they are intending to use, the library should be able to choose its own dependencies as an implementation detail given the features that they are asking for.
Contributor guide
Research direction
No Bazel source file, test, or entry point is identified in the issue. Start by studying bzlmod dependency resolution alongside the referenced nodep capability issue and Rust Cargo Features; define how additive feature sets affect conditional targets and outgoing dependencies, with Protobuf's multi-language and implementation-selection cases as the completion criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100