bazelbuild / bazelbuild/bazel

linux sandbox spawns without env variables

Open
#27,099 0 comments 0 reactions 0 assignees View on GitHub
help wanted P3 team-Local-Exec type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the bug:

There is a check to see if linux-sandbox can run on the host,

https://github.com/bazelbuild/bazel/blob/3ebd6adfdc836f456e0bcabac399f2e985c2933f/src/main/java/com/google/devtools/build/lib/sandbox/LinuxSandboxedSpawnRunner.java#L102-L123

This check invokes running `/bin/true` however on my system (nixos) this binary does not exist. Instead I have a [fuse mounted](https://github.com/Mic92/envfs) on `/bin` and `/usr/bin` to resolve all binaries under there to point to their hermetic location based on the PATH.

```
troy@troy-server ~/g/s/scuffle (main)> ls -lah /bin/true
lr----x--t 1 root root 0 Dec 31 1969 /bin/true -> /nix/store/mp7ba85zcqdj2sqwa29pql02s6nqpcxy-coreutils-9.7/bin/true
```

In my case it points to some nix-store.

and this is not a static binary.

```
troy@troy-server ~/g/s/scuffle (main)> ldd /bin/true
linux-vdso.so.1 (0x00007f4a9b7a8000)
libacl.so.1 => /nix/store/pi4h9797ldv1435a75246lwpdgmxiabk-acl-2.3.2/lib/libacl.so.1 (0x00007f4a9b797000)
libattr.so.1 => /nix/store/072g4zhidpalbdjvnslfd89nx4j9s319-attr-2.5.2/lib/libattr.so.1 (0x00007f4a9b78f000)
libgmp.so.10 => /nix/store/rbb1x89qn6h1c3r5hfz5an2h4dva8y2f-gmp-with-cxx-6.3.0/lib/libgmp.so.10 (0x00007f4a9b6e4000)
libc.so.6 => /nix/store/zdpby3l6azi78sl83cpad2qjpfj25aqx-glibc-2.40-66/lib/libc.so.6 (0x00007f4a9b400000)
/nix/store/zdpby3l6azi78sl83cpad2qjpfj25aqx-glibc-2.40-66/lib/ld-linux-x86-64.so.2 => /nix/store/zdpby3l6azi78sl83cpad2qjpfj25aqx-glibc-2.40-66/lib64/ld-linux-x86-64.so.2 (0x00007f4a9b7aa000)
troy@troy-server ~/g/s/scuffle (main)>
```

```
[pid 199553] execve("/home/troy/.cache/bazel/_bazel_troy/install/1093707fde95f200c136d266696e04f7/linux-sandbox", ["/home/troy/.cache/bazel/_bazel_troy/install/1093707fde95f200c136d266696e04f7/linux-sandbox", "-T", "15", "--", "/bin/true"], []) = 0
strace: Process 199554 attached
strace: Process 199555 attached
strace: Process 199556 attached
strace: Process 199557 attached
strace: Process 199558 attached
[pid 199558] execve("/bin/true", ["/bin/true"], ["LD_LIBRARY_PATH="]) = -1 ENOENT (No such file or directory)
[pid 199558] exit_group(1) = ?
[pid 199558] +++ exited with 1 +++
[pid 199555] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=2, si_uid=1000, si_status=1, si_utime=0, si_stime=0} ---
```

### Which category does this issue belong to?

_No response_

### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Running with the following command

```
strace -f -s 4096 -v -e trace=exit_group,exit,execve -- bazelisk --client_debug --batch test //... --spawn_strategy=linux-sandbox --nocache_test_results 2> out.txt
```

However you will need to be on a NixOS setup.

### Which operating system are you running Bazel on?

NixOs

### What is the output of `bazel info release`?

release 8.3.1

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

---- This also happens on the latest master commit 3ebd6adfdc836f456e0bcabac399f2e985c2933f

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?

```text

```

### If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

_No response_

### Have you found anything relevant by searching the web?

https://github.com/bazelbuild/bazel/issues/13994
https://github.com/bazelbuild/bazel/issues/14734

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with src/main/java/com/google/devtools/build/lib/sandbox/LinuxSandboxedSpawnRunner.java, especially the linked host-capability check around lines 102-123, and reproduce on NixOS using the provided strace command with --spawn_strategy=linux-sandbox. Trace how /bin/true is launched and how its environment is constructed; done means the linux-sandbox check works with envfs-backed binaries and does not lose the required environment.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, linux
Domain
build-system, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.