bazelbuild / bazelbuild/remote-apis

Add FetchDirectoryStream to save the extra GetTree call

Open
#355 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
445
Forks
141
PR merge metrics
No merged PRs in 30d

Description

Currently in Remote Asset API, we have FetchDirectory with the following

```protobuf
rpc FetchDirectory(FetchDirectoryRequest) returns (FetchDirectoryResponse) {
option (google.api.http) = { post: "/v1/{instance_name=**}/assets:fetchDirectory" body: "*" };
}

...

message FetchDirectoryResponse {
google.rpc.Status status = 1;

string uri = 2;

repeated Qualifier qualifiers = 3;

google.protobuf.Timestamp expires_at = 4;

// The result of the fetch, if the status had code `OK`.
// the root digest of a directory tree, suitable for fetching via
// [ContentAddressableStorage.GetTree].
build.bazel.remote.execution.v2.Digest root_directory_digest = 5;

build.bazel.remote.execution.v2.DigestFunction.Value digest_function = 6;
}
```

Note how the client will have to call GetTree with the returned digest to get the whole tree, which is an additional round trip

```protobuf
rpc GetTree(GetTreeRequest) returns (stream GetTreeResponse) {
option (google.api.http) = { get: "/v2/{instance_name=**}/blobs/{root_digest.hash}/{root_digest.size_bytes}:getTree" };
}
```

I wonder if we can simply return the `GetTreeResponse`, at least the first page, directly in FetchDirectoryResponse. That can simplify the call path for clients who are expecting a small and relatively shallow tree.

Otherwise, perhaps a `FetchDirectoryStream` variant that returns a stream of Directory messages of the whole tree would be nice to have as well.

Contributor guide

Open the contributing guide

Research direction

Start with the FetchDirectory and GetTree RPC definitions shown in the issue, comparing their response and streaming semantics. No files or tests are named. Done would require choosing and specifying one concrete API shape, including its response behavior, rather than leaving both alternatives open.

Written by the indexing model from the issue text.

Assessment

Domain
api, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.