bazelbuild / bazelbuild/bazel

Include execution cache hit ratios

Open
#16,043 6 comments 0 reactions 1 assignee Claimed by @coeuvre View on GitHub
P2 team-Remote-Exec type: feature request
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the feature request:

It would be ideal if we could surface cache hit ratios as first class information. As it stands now I'm only aware of two ways to try and calculate cache hit ratios from a users point of view. One is to create the execution log and parse it for `remoteCacheable` and `remoteCacheHit` then calculate the standard hit ratio. The other is to use build event logs and pluck out the [runner information](https://github.com/bazelbuild/bazel/blob/5.1.0/src/main/java/com/google/devtools/build/lib/buildeventstream/proto/build_event_stream.proto#L861), for example `INFO: 1436 processes: 977 disk cache hit, 456 internal, 1 darwin-sandbox, 2 worker`. As discussed in https://bazelbuild.slack.com/archives/C01E7TH8XK9/p1659330964512159?thread_ts=1659329231.807649&cid=C01E7TH8XK9 there are some heuristics we could use to assume some knowledge of cache hit ratio, but it is not concrete.

The request here is to surface cache hit ratio for different dimensions
* Cache hit for overall execution and some indication that no execution was done or 100% cache hit because no execution was done, in the case where rebuild yields nothing new.
* Cache hit for remote cached requests for different remote caches (remote/disk)
* Ratio of where cache was found, remote/disk.

### What underlying problem are you trying to solve with this feature?

We are trying to understand when we hit cache regressions from a client side point of view. Recently bazel was released with broken caching in 5.2.0 https://github.com/bazelbuild/bazel/issues/15682#issuecomment-1175781129, we (twitter) realized it when our remote build cache systems experienced a lower cache hit ratio on cache requests over the network. It would have been ideal for us to also log that kind of metrics from the clients point of view as the source of truth that the runner indeed did miss and what state it might have been in at that time. It would also allow us to put monitoring metrics around the health of the runners when executing on some branches, such as `master`.

Easy access to this information would greatly help us identify regressions sooner and give us more confidence in bazel and rules upgrades.

### Which operating system are you running Bazel on?

osx,linux

### What is the output of `bazel info release`?

_No response_

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

_No response_

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.