bazelbuild / bazelbuild/bazel

No local fallback after cache timeout

Open
#20,123 15 comments 0 reactions 0 assignees View on GitHub
not stale P3 team-Remote-Exec type: feature request
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

While running a build our Artifactory HTTP cache timed out for a request. Obviously we're looking at why it did that but we expected Bazel to fall back to running the action locally and instead it failed the build
`
ERROR: //BUILD:1802:10: Compiling .c failed: unable to finalize action: Download of '//cas/b8f31e5fda95495273a86cc5c7395298eb321490395ca90815a9184f2a9ec980' timed out. Received 0 bytes.
`

The documentation suggests `--remote_local_fallback` only applies to remote execution but some comments on the bug tracker suggested it might also apply to remote caching so we tried that but still saw the issue.

I can try to recreate the issue but it's not totally trivial as I'll need to setup an HTTP server that can deliberately time out. So thought I'd check if this is expected behavior or if perhaps there is a trivially obvious bug to someone who knows the Bazel source.

### Which category does this issue belong to?

Core

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Not trivial to reproduce! I can work on that if it would be useful.

### Which operating system are you running Bazel on?

Linux - RHEL 8

### What is the output of `bazel info release`?

release 7.0.0-pre.20231011.2- (@non-git)

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

Built from the 7.0.0-pre.20231011.2 release tag.

### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

_No response_

### Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

_No response_

### Have you found anything relevant by searching the web?

No

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by tracing how remote cache download timeouts are handled and how --remote_local_fallback is documented for remote execution versus remote caching. If possible, reproduce the timeout with a deliberately unresponsive HTTP cache, then identify the expected fallback behavior and add focused regression coverage. Done means the behavior is confirmed or corrected with a test.

Written by the indexing model from the issue text.

Assessment

Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.