bazelbuild / bazelbuild/bazel

Send overridden values for a key in `BuildMetadataEvent` via a new submessage.

Open
#18,517 5 comments 1 reaction 0 assignees View on GitHub
team-Core type: feature request untriaged
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the feature request:

Currently, `--build_metadata` is converted to a `BuildMetadataEvent` by [merging the key-value pairs through a map](https://github.com/bazelbuild/bazel/blob/10fd043243e9df7be026783758cfdb4c67c1ccd9/src/main/java/com/google/devtools/build/lib/runtime/CommandEnvironment.java#L757). This means that the command line `--build_metadata=FOO=abc --build_metadata=FOO=def` yields a final `BuildMetadataEvent` containing only `'FOO': 'def'`. This makes it difficult to, for example, support tag-like metadata that accumulates labels via `config` settings in `.bazelrc`. This feature request is admittedly a little niche.

Remote-related flags like `remote_header` merge down to a comma-separated list, but obviously we can't do this in-place because it would break backward compatability.

By adding a new submessage to `BuildMetadataEvent` for overridden values, we could convey these values to external tools without breaking backward-compatability. Roughly something like this:

```
message BuildMetadata {
// Custom metadata for the build. If the user tried to set a given key
// multiple times, this will contain the *last* value that they set.
map metadata = 1;

// Represents the list of additional metadata flag values for a given metadata
// key. This will be set if two different --build_metadata arguments try to
// set the same key, and will contain all values set for the key *except* the
// last value set. Whether and how these values are used is up to the
// external tool consuming the metadata.
message ExtraValues {
repeated string values = 1;
}
map extra_values = 2;
}
```

I have a [PR for this drafted](https://github.com/bazelbuild/bazel/compare/master...jdhollen:bazel:overridden-metadata?expand=1) because it's kinda easier to code than to describe. I'd be happy to clean this up / add tests / send the PR if this approach seems fine to folks.

### What underlying problem are you trying to solve with this feature?

I'm trying to implement a feature that uses `BuildMetadataEvent` to specify tags on an invocation ("tags" in the general sense, not the target attribute sense). This would be something like `my-teams-ci-run`, or `secret-builds`, or `release-2023-05-26_01`. `BuildMetadataEvent` is convenient for this, but it can be a pain to try to aggregate all of the potential tag values prior to invoking bazel--it'd be nice to just specify `MY_TAGS=foo`, `MY_TAGS=bar`, etc., and forget about concatenating them myself.

The proposal above seems simple enough to me, but obviously there are workarounds that function, too:

- Aggregate tags before the initial bazel invocation (i.e., a wrapper script).
- Do some kind of aggregation in environment variables (i.e., also a wrapper script).
- Try to hoover up the data from `StructuredCommandLine` or similar, since it's there (this seems less desirable since we already have an event that parses `build_metadata`)

### Which operating system are you running Bazel on?

N/A

### What is the output of `bazel info release`?

N/A

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

N/A

### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

```text
N/A
```

### Have you found anything relevant by searching the web?

Tried hunting on Slack, mailing list, and issues, but didn't turn up any past discussions (not terribly surprising).

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start in src/main/java/com/google/devtools/build/lib/runtime/CommandEnvironment.java around line 757 and inspect how BuildMetadataEvent is assembled. Review the drafted overridden-metadata change and the existing BuildMetadata definition; done means preserving the current metadata map while exposing earlier repeated values through the proposed extra_values submessage, with tests added for repeated keys.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.