bazelbuild / bazelbuild/bazel

Symlinks inside of TreeArtifacts fail with RBE and BwoB

Open
#22,504 8 comments 0 reactions 0 assignees View on GitHub
not stale team-Remote-Exec type: support / not a bug (process)
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

With rules_oci 2.x we implemented a feature where we create symlinks inside of a tree-artifact that points out of the tree-artifact (https://github.com/bazelbuild/bazel/issues/20891) to files created by other actions and this works fine with `RBE` and `Local` execution.

Note that the files that symlinks point to are not in the inputs of the action that creates the symlink, and i don't think it matters much.

However, when the same action run on RBE with `--remote_download_minimal` flag, it fails with the message below.

Here's the Bazel slack channel threads which might be helpful: https://aspect-build.slack.com/archives/C04281DTLH0/p1716397067812139

### Which category does this issue belong to?

_No response_

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

1- clone https://github.com/bazel-contrib/rules_oci/tree/2.x
2- run bazel test //examples/... --remote_download_minimal on RBE

### Which operating system are you running Bazel on?

Darwin

### What is the output of `bazel info release`?

7.1.1

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?

_No response_

### Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

I don't think is a regression.

### Have you found anything relevant by searching the web?

Probably related https://github.com/bazelbuild/bazel/issues/15454

### Any other information, logs, or outputs that you want to share?

```ERROR: /workspaces/infrastructure/services/bazel/buildbarn/asset/BUILD.bazel:10:6: error while validating output tree artifact services/bazel/buildbarn/asset/asset.image: child blobs/sha256/38fb3dea748fd6d3f00ff8209e3595127b39da449b0b8d51c0e98bb16421ef5c is a dangling symbolic link
ERROR: /workspaces/infrastructure/services/bazel/buildbarn/asset/BUILD.bazel:10:6: OCI Image //services/bazel/buildbarn/asset:asset.image failed: not all outputs were created or valid```

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure from rules_oci 2.x with `bazel test //examples/... --remote_download_minimal` on RBE, using the reported Bazel 7.1.1 setup. Start with the output-tree validation for `services/bazel/buildbarn/asset/asset.image` and the dangling link error in `services/bazel/buildbarn/asset/BUILD.bazel`. Done means the symlinked tree artifact validates and the OCI image test succeeds under minimal remote downloads.

Written by the indexing model from the issue text.

Assessment

Domain
build-system, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.