bazelbuild / bazelbuild/bazel

Uploading to disk cache should succeed even when uploading to remote cache fails (or vice-versa)

Open
#24,073 6 comments 0 reactions 1 assignee Claimed by @tjgq View on GitHub
P2 team-Remote-Exec type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

We have two use cases where we need to push existing local build outputs to a remote cache:

1) We have a set of caches around the globe, and need to push each nightly build from the (single) build server to all of the caches. The caches do not have their own synchronization.
2) If the cache goes down during a nightly build, we need to allow the nightly build to continue to success locally, and populate the cache once it comes up again.

I can't see any feature in bazel that would allow this. It would be massively helpful for us, and we could perhaps make a contribution if it could be aligned to bazel's architecture.

I'm imagining this extra option `--remote_upload_existing_results=true`
```
bazel build //... --remote_cache=... --remote_upload_local_results=true --remote_upload_existing_results=true
```

Behaviour with `remote_upload_existing_results=true`:
- if bazel chooses to short circuit an action due to the action result already existing on disk with correct checksum, the action and action result are still sent to the cache if missing.

This could be integrated into bazel or developed as a parallel tool.

### Which category does this issue belong to?

Remote Execution

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

_No response_

### Which operating system are you running Bazel on?

windows

### What is the output of `bazel info release`?

7.3.1

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?

_No response_

### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

_No response_

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.