Send overridden values for a key in `BuildMetadataEvent` via a new submessage.
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the feature request:
Currently, `--build_metadata` is converted to a `BuildMetadataEvent` by [merging the key-value pairs through a map](https://github.com/bazelbuild/bazel/blob/10fd043243e9df7be026783758cfdb4c67c1ccd9/src/main/java/com/google/devtools/build/lib/runtime/CommandEnvironment.java#L757). This means that the command line `--build_metadata=FOO=abc --build_metadata=FOO=def` yields a final `BuildMetadataEvent` containing only `'FOO': 'def'`. This makes it difficult to, for example, support tag-like metadata that accumulates labels via `config` settings in `.bazelrc`. This feature request is admittedly a little niche.
Remote-related flags like `remote_header` merge down to a comma-separated list, but obviously we can't do this in-place because it would break backward compatability.
By adding a new submessage to `BuildMetadataEvent` for overridden values, we could convey these values to external tools without breaking backward-compatability. Roughly something like this:
```
message BuildMetadata {
// Custom metadata for the build. If the user tried to set a given key
// multiple times, this will contain the *last* value that they set.
map metadata = 1;
// Represents the list of additional metadata flag values for a given metadata
// key. This will be set if two different --build_metadata arguments try to
// set the same key, and will contain all values set for the key *except* the
// last value set. Whether and how these values are used is up to the
// external tool consuming the metadata.
message ExtraValues {
repeated string values = 1;
}
map extra_values = 2;
}
```
I have a [PR for this drafted](https://github.com/bazelbuild/bazel/compare/master...jdhollen:bazel:overridden-metadata?expand=1) because it's kinda easier to code than to describe. I'd be happy to clean this up / add tests / send the PR if this approach seems fine to folks.
### What underlying problem are you trying to solve with this feature?
I'm trying to implement a feature that uses `BuildMetadataEvent` to specify tags on an invocation ("tags" in the general sense, not the target attribute sense). This would be something like `my-teams-ci-run`, or `secret-builds`, or `release-2023-05-26_01`. `BuildMetadataEvent` is convenient for this, but it can be a pain to try to aggregate all of the potential tag values prior to invoking bazel--it'd be nice to just specify `MY_TAGS=foo`, `MY_TAGS=bar`, etc., and forget about concatenating them myself.
The proposal above seems simple enough to me, but obviously there are workarounds that function, too:
- Aggregate tags before the initial bazel invocation (i.e., a wrapper script).
- Do some kind of aggregation in environment variables (i.e., also a wrapper script).
- Try to hoover up the data from `StructuredCommandLine` or similar, since it's there (this seems less desirable since we already have an event that parses `build_metadata`)
### Which operating system are you running Bazel on?
N/A
### What is the output of `bazel info release`?
N/A
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
N/A
### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?
```text
N/A
```
### Have you found anything relevant by searching the web?
Tried hunting on Slack, mailing list, and issues, but didn't turn up any past discussions (not terribly surprising).
### Any other information, logs, or outputs that you want to share?
_No response_
Contributor guide
Research direction
Start in src/main/java/com/google/devtools/build/lib/runtime/CommandEnvironment.java around line 757 and inspect how BuildMetadataEvent is assembled. Review the drafted overridden-metadata change and the existing BuildMetadata definition; done means preserving the current metadata map while exposing earlier repeated values through the proposed extra_values submessage, with tests added for repeated keys.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100