collect2: fatal error: cannot find 'ld' with build --incompatible_strict_action_env
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the bug:
Seems to be related to #20834
When I enable `build --incompatible_strict_action_env` in .bazelrc building our project fails with the following error:
```
➜ kotlin git:(agluszak/refactor-spaghetti) ✗ bazel build //... --sandbox_debug
INFO: Analyzed 150 targets (0 packages loaded, 7 targets configured).
DEBUG: Sandbox debug output for CppLink @@protobuf~21.7//:protoc:
1707844200.104456969: src/main/tools/linux-sandbox.cc:156: calling pipe(2)...
1707844200.104511302: src/main/tools/linux-sandbox.cc:165: Netns is 0
1707844200.104513667: src/main/tools/linux-sandbox.cc:176: calling clone(2)...
1707844200.105022932: src/main/tools/linux-sandbox.cc:185: linux-sandbox-pid1 has PID 80651
1707844200.105051827: src/main/tools/linux-sandbox-pid1.cc:700: Pid1Main started
1707844200.105319123: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/execroot -> /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-execroot
1707844200.105360231: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/execroot -> /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-working-directory
1707844200.105373727: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/external/bazel_tools -> /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-source-roots/0
1707844200.105393635: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/external/bazel_tools~cc_configure_extension~local_config_cc -> /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-source-roots/1
1707844200.105407010: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp -> /tmp
1707844200.105425235: src/main/tools/linux-sandbox-pid1.cc:311: writable: /dev/shm
1707844200.105436987: src/main/tools/linux-sandbox-pid1.cc:311: writable: /tmp
1707844200.105454781: src/main/tools/linux-sandbox-pid1.cc:311: writable: /tmp/bazel-execroot/_main
1707844200.105465631: src/main/tools/linux-sandbox-pid1.cc:327: working dir: /tmp/bazel-working-directory/_main
1707844200.105580890: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /
1707844200.105589215: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev
1707844200.105595107: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/pts
1707844200.105601248: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /dev/shm
1707844200.105606588: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/hugepages
1707844200.105612490: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/mqueue
1707844200.105619072: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run
1707844200.105625053: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/lock
1707844200.105630794: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/credentials/systemd-sysusers.service
1707844200.105636886: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/qemu
1707844200.105642567: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/user/1000
1707844200.105649129: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/user/1000/gvfs
1707844200.105692892: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/user/1000/doc
1707844200.105699324: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/docker/netns/f38912d93c2c
1707844200.105703352: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /run/docker/netns/f38912d93c2c, nullptr, 4129, nullptr) failure (Permission denied) ignored
1707844200.105712730: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys
1707844200.105718380: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/firmware/efi/efivars
1707844200.105729321: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/security
1707844200.105750030: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/cgroup
1707844200.105757975: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/pstore
1707844200.105764367: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/bpf
1707844200.105770820: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/debug
1707844200.105777412: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/tracing
1707844200.105784055: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/fuse/connections
1707844200.105791569: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/config
1707844200.105798192: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc
1707844200.105803432: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc/sys/fs/binfmt_misc
1707844200.105813982: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /proc/sys/fs/binfmt_misc, nullptr, 2101281, nullptr) failure (Operation not permitted) ignored
1707844200.105854238: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc/sys/fs/binfmt_misc
1707844200.105861712: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /recovery
1707844200.105866732: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /boot/efi
1707844200.105891278: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /var/lib/docker/overlay2/f8d24e86254fc80e1069e49e0e17b4ac571ff497cb87bf0590dbd5d86ec9add3/merged
1707844200.105896468: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /var/lib/docker/overlay2/f8d24e86254fc80e1069e49e0e17b4ac571ff497cb87bf0590dbd5d86ec9add3/merged, nullptr, 2101281, nullptr) failure (Permission denied) ignored
1707844200.105899704: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/.mount_jetbrasqy1gx
1707844200.105915123: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /tmp/.mount_jetbrasqy1gx, nullptr, 2101287, nullptr) failure (No such file or directory) ignored
1707844200.105919051: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-execroot
1707844200.105925743: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-working-directory
1707844200.105950019: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-source-roots/0
1707844200.105956882: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-source-roots/1
1707844200.105962754: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp
1707844200.105967643: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-execroot
1707844200.105972612: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-working-directory
1707844200.105977221: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/0
1707844200.105982391: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/1
1707844200.105987691: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /dev/shm
1707844200.105992610: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp
1707844200.105997099: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-execroot
1707844200.106009552: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-working-directory
1707844200.106014261: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/0
1707844200.106019261: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/1
1707844200.106024090: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp/bazel-execroot/_main
1707844200.106029079: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp/bazel-working-directory/_main
1707844200.106194844: src/main/tools/linux-sandbox.cc:202: done manipulating pipes
1707844200.106516263: src/main/tools/linux-sandbox-pid1.cc:496: calling fork...
1707844200.106983849: src/main/tools/linux-sandbox-pid1.cc:533: child started with PID 2
1707844200.131119984: src/main/tools/linux-sandbox-pid1.cc:550: wait returned pid=2, status=0x100
1707844200.131138990: src/main/tools/linux-sandbox-pid1.cc:568: child exited normally with code 1
1707844200.131745710: src/main/tools/linux-sandbox.cc:243: child exited normally with code 1
Run this command to start an interactive shell in an identical sandboxed environment:
(exec env - \
PATH=/bin:/usr/bin:/usr/local/bin \
PWD=/proc/self/cwd \
TMPDIR=/tmp \
ZERO_AR_DATE=1 \
/home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/install/14fb027596f626f2526df4873ea20b8b/linux-sandbox -W /tmp/bazel-working-directory/_main -t 15 -w /dev/shm -w /tmp -w /tmp/bazel-execroot/_main -M /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/execroot -m /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-execroot -M /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/execroot -m /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-working-directory -M /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/external/bazel_tools -m /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-source-roots/0 -M /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/external/bazel_tools~cc_configure_extension~local_config_cc -m /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-source-roots/1 -M /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp -m /tmp -S /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/stats.out -D /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/debug.out -- /bin/sh -i)
ERROR: /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/external/protobuf~21.7/BUILD.bazel:457:10: Linking external/protobuf~21.7/protoc [for tool] failed: (Exit 1): linux-sandbox failed: error executing CppLink command
(cd /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/execroot/_main && \
exec env - \
PATH=/bin:/usr/bin:/usr/local/bin \
PWD=/proc/self/cwd \
TMPDIR=/tmp \
ZERO_AR_DATE=1 \
/home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/install/14fb027596f626f2526df4873ea20b8b/linux-sandbox -W /tmp/bazel-working-directory/_main -t 15 -w /dev/shm -w /tmp -w /tmp/bazel-execroot/_main -M /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/execroot -m /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-execroot -M /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/execroot -m /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-working-directory -M /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/external/bazel_tools -m /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-source-roots/0 -M /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/external/bazel_tools~cc_configure_extension~local_config_cc -m /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp/bazel-source-roots/1 -M /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/_hermetic_tmp -m /tmp -S /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/stats.out -D /home/andrzej.gluszak/.cache/bazel/_bazel_andrzej.gluszak/1c60d9a8d822a2b05451e359eb75781d/sandbox/linux-sandbox/319/debug.out -- /usr/bin/gcc @bazel-out/k8-opt-exec-ST-13d3ddad9198/bin/external/protobuf~21.7/protoc-2.params)
collect2: fatal error: cannot find 'ld'
compilation terminated.
INFO: Elapsed time: 0.190s, Critical Path: 0.04s
INFO: 2 processes: 2 internal.
ERROR: Build did NOT complete successfully
```
Code is available here: https://github.com/JetBrains/bazel-bsp/commit/8a876006fafacf53a5bcd9e574af5724b42e2189
### Which category does this issue belong to?
C++ Rules, Core, Local Execution
### What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
_No response_
### Which operating system are you running Bazel on?
Ubuntu
### What is the output of `bazel info release`?
release 7.0.2
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
_No response_
### What's the output of `git remote get-url origin; git rev-parse HEAD` ?
_No response_
### Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.
_No response_
### Have you found anything relevant by searching the web?
_No response_
### Any other information, logs, or outputs that you want to share?
_No response_
Contributor guide
Research direction
Reproduce the failure with `bazel build //... --incompatible_strict_action_env --sandbox_debug`, then inspect `.bazelrc` and the external protobuf target reported at `BUILD.bazel:457:10`. Compare the sandbox environment and linker lookup with and without the strict action environment. Done means the build completes without the `cannot find 'ld'` error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, linux
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100