bazelbuild / bazelbuild/bazel

Bazel digest calculation of big sparse files is slow

Open
#25,265 2 comments 0 reactions 0 assignees View on GitHub
help wanted P3 team-Performance type: feature request
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

We use Bazel to build partition and disk images. The individual partitions are built across multiple build steps and assembled into a disk image in the end.

The images are stored as sparse files in the filesystem and their disk size is 1-10% of their logical size. For example: 100GB size but only 2GB on disk. When Bazel computes the digest of the images, it cannot take advantage of the sparse nature of the file (e.g. by using SEEK_HOLE and SEEK_DATA) and reads the giant holes which is very slow.

While DigestFunction in [remote_execution.proto](https://cs.opensource.google/bazel/bazel/+/master:third_party/remoteapis/build/bazel/remote/execution/v2/remote_execution.proto;l=1861) does have a few digest functions that could be easily implemented to work optimally with sparse files, these are not available in the [hashFunctionRegistry](https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/vfs/DigestHashFunction.java;l=41;drc=0d0c991da0f2c0de8135249605c58203e6d7641f). Only BLAKE3, SHA1 and SHA256 are available.

### Which category does this issue belong to?

Performance

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

```
genrule(
name = "big_sparse_file",
outs = ["big_sparse_file.dat"],
cmd = "truncate $@ -s 10G",
)
```

```INFO: Elapsed time: 23.773s, Critical Path: 14.13s```

### Which operating system are you running Bazel on?

Ubuntu 20.04

### What is the output of `bazel info release`?

release 7.4.1

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?

_No response_

### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

_No response_

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with src/main/java/com/google/devtools/build/lib/vfs/DigestHashFunction.java and the referenced DigestFunction definitions in remote_execution.proto. Reproduce the behavior with the provided genrule on Ubuntu 20.04 and compare digesting a large sparse file with its reported disk usage. Done means the sparse-file digest path no longer reads giant holes and the example completes substantially faster.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, linux
Domain
build-system, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.