bazelbuild / bazelbuild/bazel

remote exceptions: surface the retry attempts and action id

Open
#18,161 1 comment 0 reactions 1 assignee Claimed by @coeuvre View on GitHub
P2 team-Remote-Exec type: feature request
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the feature request:

Remote failures would be easier to debug if failure messages included:

- the remote action id
- the number of retry attempts.

Example:

```
remote spawn failed: RESOURCE_EXHAUSTED: server out of workers

```

would become:

```
remote spawn failed: RESOURCE_EXHAUSTED: server out of workers (action_id: , retries: 5/5)

```

### What underlying problem are you trying to solve with this feature?

Simplify debugging remote failures:

- without the action_id, it's difficult for the remote server owner to locate the problem.
- without the retry information, it may be unclear what actually happened. Real life example that prompted this: we encountered an error that clearly should have been retriable, but appears it was not retried. It is hard to locate the bug without the retry context info.

### Which operating system are you running Bazel on?

linux,mac

### What is the output of `bazel info release`?

_No response_

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

_No response_

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.