bazelbuild / bazelbuild/remote-apis

REv3 idea: do stuff other than execve()

Open
#198 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
445
Forks
141
PR merge metrics
No merged PRs in 30d

Description

What's pretty awesome about the Remote Execution protocol is that you can seamlessly send requests to multiple platforms. Run a couple of commands on Linux, combine those results into an action scheduled on Windows, etc.

One common pattern that people have is that they use run of the mill x86 hardware to build embedded firmware images. Those may then be flashed onto an embedded system (e.g., a bare-metal ARM development board) for integration tests. What's a bit annoying right now is that the Remote Execution protocol has no native way to describe such workloads. The best you can achieve is that you schedule actions on, say, a Linux box ("host system") that's attached to the embedded system, and that your test's entry point is a wrapper script that runs on the host system, calling into tooling to do the flashing, rebooting of the hardware, capturing serial I/O, etc.. Though that approach 'works', there are some disadvantages:

- Security: The wrapper script may contain arbitrary commands. It's hard for administrators of such host systems to ensure that the engineers on the project don't make a mess out of the system.
- Cost/scaling: To save costs, you may want to attach the host system to a larger number of embedded boards. Now you need to make 100% sure that nobody ever makes changes to the wrapper script that cause it to interact with the wrong board, as that may introduce flakiness for everyone else.
- Compatibility: It's pretty easy to formalise what needs to be flashed onto a certain kind of embedded board. Usually it's a list of n firmware images that need to be flashed into different ROMs. If you use a wrapper script, you now also need to care about how the host system is provisioned. It may be hard to provide compatibility guarantees for that over time.

One thing to consider is to make Command (and parts of Action?) 'pluggable'. The version of Command we have right now is a good fit for UNIX-like systems. For running tests on embedded boards, you may want to use a different message:

```
message MyAwesomeEmbeddedDeviceCommand {
build.bazel.remote.execution.v3.Digest boot_rom = 1;
build.bazel.remote.execution.v3.Digest wifi_chip_firmware = 2;
build.bazel.remote.execution.v3.Digest gpu_firmware = 3;
build.bazel.remote.execution.v3.Digest storage_controller_firmware = 4;
}
```

Or if it's an embedded Linux board (that is capable of eventually running regular UNIX commands), you may want to do something as creative as:

```
message MyAwesomeEmbeddedLinuxDeviceCommand {
build.bazel.remote.execution.v3.Digest linux_rom = 1;
build.bazel.remote.execution.v3.Command command_to_run_inside_of_linux_after_it_has_booted = 2;
}
```

----------------------------------------------

Footnote: An approach like this raises the question: What happens if I declare a custom command that looks like this?
```
message FetchCommand {
repeated string urls = 1;
build.bazel.remote.execution.v3.Digest expected_digest = 2;
}
```
Would that make the Remote Asset API superfluous?

Contributor guide

Open the contributing guide

Research direction

Start by reading the REv3 Command and Action definitions and the Remote Asset API referenced in the issue. Compare their current assumptions with the embedded-device examples and determine whether a pluggable command design is warranted. Done requires an agreed protocol design and scope, not just an isolated code edit.

Written by the indexing model from the issue text.

Assessment

Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.