Sharing --repository_cache with multiple users results in cache misses and overrides of entries
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the problem / feature request:
I would like to set a system-wide local path for the repository cache (--repository_cache) so multiple users on the same system can avoid having to re-download the same external dependency. This implies setting the repository cache directory to be group readable and writable, in addition to setting the SGID bit, so new files are owned by the group that is common to all users. This allows everyone to still be able to read from the cache despite which particular user generated the hash entries when running bazel. World readable/writable could also be used.
However, when an entry of interest at repository_cache_dir/content_addressable/sha256/{hash}/file is owned by a different user, bazel treats it as a cache miss, and causes a re-download of the external dependency and overwrites the existing entry with the new file but owned by the user that ran bazel.
### Feature requests: what underlying problem are you trying to solve with this feature?
Trying to avoid each user on a particular system to have to re-download an external dependency when there's already an entry for it in the repository cache directory specified by the `--repository_cache` flag, while also avoiding copying the dependency to each user's bazel output base directory by using the `--experimental_repository_cache_hardlinks` flag in order to save disk space.
### Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
**.bazelrc**
```
build --experimental_repository_cache_hardlinks
build --repository_cache=/path/to/bazel-rep-cache
build --disk_cache=/path/to/bazel-disk-cache
```
**WORKSPACE**
```
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file")
http_file(
name = "bazel_source_code",
urls=["https://github.com/bazelbuild/bazel/archive/refs/tags/4.1.0.tar.gz"],
sha256="27b887da66789638d904cb3cd3526418145e1f2f1e2891209f12d60f566850ff",
)
```
**BUILD**
```
genrule(
name="bazel_source_file_paths",
srcs=[
"@bazel_source_code//file",
],
outs=["bazel_source_file_paths.txt"],
cmd="tar -tf $(SRCS) > $@",
)
```
1. `bazel build :bazel_source_file_paths`
2. `stat $(bazel info output_base)/external/bazel_source_code/file/downloaded` # note that it's a regular file
3. `bazel clean --expunge` # needed to cause a hardlink generation on subsequent bazel invocations.
4. `bazel build :bazel_source_file_paths` # will result in a cache hit this time and create a hardlink
5. `stat $(bazel info output_base)/external/bazel_source_code/file/downloaded` # note that it's a hardlink file
6. `sudo chown -R root:root /path/to/bazel-rep-cache` # simulate entries generated by another user
7. `sudo chmod -R o+r /path/to/bazel-rep-cache` # allow all to read. One could use a common group instead but simplifying for this example
8. `sudo chmod o+w /path/to/bazel-rep-cache /content_addressable/sha256/27b887da66789638d904cb3cd3526418145e1f2f1e2891209f12d60f566850ff` # allow all to write
9. `bazel clean --expunge`
10. `bazel build :bazel_source_file_paths` # ideally should result in a cache hit again since entry still exists, but owned by a different user (`root` in this case now), but it doesn't. Causes re-download and an overwrite of cache entry. Noticed as well that it creates a `tmp-...` file inside the `/path/to/bazel-rep-cache/content_addressable/sha256/{hash}` directory first before overwriting original file.
### What operating system are you running Bazel on?
RedHat 7.9
### What's the output of `bazel info release`?
release 3.6.0
### Have you found anything relevant by searching the web?
https://docs.bazel.build/versions/main/guide.html#repository-cache
https://bazel.build/designs/2016/09/30/repository-cache.html
https://groups.google.com/g/bazel-discuss/c/mzEXZMCa2Ic
https://github.com/bazelbuild/bazel/issues/8496
Contributor guide
Assessment
This issue has not been assessed yet.