Building rust 1.75.0 fails on NetBSD when using an external LLVM due to missing RPATH

Code

I am trying to keep up with rust releases, cross-building bootstrap kits for the various NetBSD targets using cross-compilation from a NetBSD/amd64 host. While doing this, I have been using the rust-supplied amd64 binary bootstrap kit to bootstrap the amd64 rust compiler. That has worked up until 1.74.1 (using the 1.73.0 bootstrap kit), but now fails using the 1.74.1 bootstrap kit to build 1.75.0.

I expected to see this happen: that the build would succeed.

Instead, this happened: The build progresses until "stage1" as far as I can see. Here is the part of the build log which surrounds the build error:

     Running `/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/bootstrap/debug/rustc --crate-name rustc_main --edition=2021 compiler/rustc/src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C opt-level=3 -C embed-bitcode=no --cfg 'feature="llvm"' --cfg 'feature="max_level_info"' -Zunstable-options --check-cfg 'values(feature, "jemalloc", "jemalloc-sys", "llvm", "max_level_info", "rustc_use_parallel_compiler")' --check-cfg 'names()' --check-cfg 'values()' -C metadata=99ab614842099465 -C extra-filename=-99ab614842099465 --out-dir /usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage0-rustc/x86_64-unknown-netbsd/release/deps --target x86_64-unknown-netbsd -L dependency=/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage0-rustc/x86_64-unknown-netbsd/release/deps -L dependency=/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage0-rustc/release/deps --extern rustc_codegen_ssa=/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage0-rustc/x86_64-unknown-netbsd/release/deps/librustc_codegen_ssa-f10772593a84d8ca.rlib --extern rustc_driver=/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage0-rustc/x86_64-unknown-netbsd/release/deps/librustc_driver-d78e0abcd3c75aa5.so --extern rustc_driver_impl=/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage0-rustc/x86_64-unknown-netbsd/release/deps/librustc_driver_impl-df6994cba8710313.rlib --extern rustc_smir=/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage0-rustc/x86_64-unknown-netbsd/release/deps/librustc_smir-7fee53f16ca15c1a.rlib --extern stable_mir=/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage0-rustc/x86_64-unknown-netbsd/release/deps/libstable_mir-9381a3ae56e93b67.rlib --cfg=bootstrap --cfg=windows_raw_dylib -Csymbol-mangling-version=v0 -Zunstable-options '--check-cfg=values(bootstrap)' '--check-cfg=values(parallel_compiler)' '--check-cfg=values(no_btreemap_remove_entry)' '--check-cfg=values(crossbeam_loom)' '--check-cfg=values(span_locations)' '--check-cfg=values(rustix_use_libc)' '--check-cfg=values(emulate_second_only_system)' '--check-cfg=values(windows_raw_dylib)' -Zmacro-backtrace -Clink-args=-Wl,-z,origin '-Clink-args=-Wl,-rpath,$ORIGIN/../lib' -Zunstable-options -Csplit-debuginfo=off -Zunstable-options '-Wrustc::internal' -Cprefer-dynamic -Z binary-dep-depinfo -L native=/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage0-rustc/x86_64-unknown-netbsd/release/build/psm-2407b8a67950d553/out -L native=/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage0-rustc/x86_64-unknown-netbsd/release/build/rustc_llvm-13a2ebc4724c6674/out -L native=/usr/pkg/lib -L native=/usr/lib -L native=/usr/pkg/lib`
    Finished release [optimized] target(s) in 4m 52s
Creating a sysroot for stage1 compiler (use `rustup toolchain link 'name' build/host/stage1`)
Removing sysroot /usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage1 to avoid caching bugs
Removing sysroot /usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage1/lib/rustlib/x86_64-unknown-netbsd/lib to avoid caching bugs
Dirty - /usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage1-std
Building stage1 library artifacts (x86_64-unknown-netbsd)
running: cd "/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src" && env -u MAKEFLAGS -u MFLAGS AR_x86_64_unknown_netbsd="ar" CARGO_INCREMENTAL="0" CARGO_PROFILE_RELEASE_DEBUG="0" CARGO_PROFILE_RELEASE_DEBUG_ASSERTIONS="false" CARGO_PROFILE_RELEASE_OVERFLOW_CHECKS="false" CARGO_TARGET_DIR="/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage1-std" CC_x86_64_unknown_netbsd="gcc" CFG_COMPILER_HOST_TRIPLE="x86_64-unknown-netbsd" CFG_DISABLE_UNSTABLE_FEATURES="1" CFG_RELEASE_CHANNEL="stable" CFLAGS_x86_64_unknown_netbsd="-O2 -ffunction-sections -fdata-sections -fPIC -m64 -O2 -I/usr/pkg/include -I/usr/include -I/usr/pkg/include/python3.11" CXXFLAGS_x86_64_unknown_netbsd="-O2 -ffunction-sections -fdata-sections -fPIC -m64 -O2 -I/usr/pkg/include -I/usr/include -I/usr/pkg/include/python3.11" CXX_x86_64_unknown_netbsd="c++" LIBC_CHECK_CFG="1" RANLIB_x86_64_unknown_netbsd="ar s" REAL_LIBRARY_PATH="/usr/pkgsrc/wip/rust175/work/rust-bootstrap/lib" REAL_LIBRARY_PATH_VAR="LD_LIBRARY_PATH" RUSTBUILD_NATIVE_DIR="/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/native" RUSTC="/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/bootstrap/debug/rustc" RUSTC_BOOTSTRAP="1" RUSTC_BREAK_ON_ICE="1" RUSTC_ERROR_METADATA_DST="/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/tmp/extended-error-metadata" RUSTC_FORCE_UNSTABLE="1" RUSTC_HOST_FLAGS="-Zunstable-options --check-cfg=values(bootstrap)" RUSTC_INSTALL_BINDIR="bin" RUSTC_LIBDIR="/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage1/lib" RUSTC_LINT_FLAGS="-Wrust_2018_idioms -Wunused_lifetimes -Wsemicolon_in_expressions_from_macros -Dwarnings" RUSTC_REAL="/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc" RUSTC_SNAPSHOT="/usr/pkgsrc/wip/rust175/work/rust-bootstrap/bin/rustc" RUSTC_SNAPSHOT_LIBDIR="/usr/pkgsrc/wip/rust175/work/rust-bootstrap/lib" RUSTC_STAGE="1" RUSTC_SYSROOT="/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage1" RUSTC_VERBOSE="1" RUSTDOC="/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/bootstrap/debug/rustdoc" RUSTDOCFLAGS="-Csymbol-mangling-version=legacy -Zunstable-options -Zunstable-options --check-cfg=values(bootstrap) --check-cfg=values(stdarch_intel_sde) --check-cfg=values(no_fp_fmt_parse) --check-cfg=values(no_global_oom_handling) --check-cfg=values(no_rc) --check-cfg=values(no_sync) --check-cfg=values(freebsd12) --check-cfg=values(freebsd13) --check-cfg=values(backtrace_in_libstd) --check-cfg=values(target_env,\"libnx\") --check-cfg=values(target_arch,\"asmjs\",\"spirv\",\"nvptx\",\"xtensa\") -Dwarnings -Wrustdoc::invalid_codeblock_attributes --crate-version 1.75.0\t(82e1608df\t2023-12-21)\t(built\tfrom\ta\tsource\ttarball) -Zcrate-attr=doc(html_root_url=\"https://doc.rust-lang.org/1.75.0/\") -Zcrate-attr=warn(rust_2018_idioms)" RUSTDOC_REAL="/path/to/nowhere/rustdoc/not/required" RUSTFLAGS="-Csymbol-mangling-version=legacy -Zunstable-options -Zunstable-options --check-cfg=values(bootstrap) --check-cfg=values(stdarch_intel_sde) --check-cfg=values(no_fp_fmt_parse) --check-cfg=values(no_global_oom_handling) --check-cfg=values(no_rc) --check-cfg=values(no_sync) --check-cfg=values(freebsd12) --check-cfg=values(freebsd13) --check-cfg=values(backtrace_in_libstd) --check-cfg=values(target_env,\"libnx\") --check-cfg=values(target_arch,\"asmjs\",\"spirv\",\"nvptx\",\"xtensa\") -Zmacro-backtrace -Clink-args=-Wl,-z,origin -Clink-args=-Wl,-rpath,$ORIGIN/../lib -Zunstable-options -Csplit-debuginfo=off -Cprefer-dynamic -Zinline-mir -Cembed-bitcode=yes -Zcrate-attr=doc(html_root_url=\"https://doc.rust-lang.org/1.75.0/\")" RUST_COMPILER_RT_ROOT="/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/src/llvm-project/compiler-rt" RUST_TEST_THREADS="32" WINAPI_NO_BUNDLED_LIBRARIES="1" __CARGO_DEFAULT_LIB_METADATA="stablestd" "/usr/pkgsrc/wip/rust175/work/rust-bootstrap/bin/cargo" "build" "--target" "x86_64-unknown-netbsd" "--release" "-Zcheck-cfg=names,values,output" "-Zbinary-dep-depinfo" "-j" "32" "-v" "--frozen" "--features" " panic-unwind backtrace compiler-builtins-c" "--manifest-path" "/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/library/sysroot/Cargo.toml" "--message-format" "json-render-diagnostics"
error: process didn't exit successfully: `/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/bootstrap/debug/rustc -vV` (exit status: 1)
--- stdout

Did not run successfully: exit status: 1
LD_LIBRARY_PATH="/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage1/lib:/usr/pkgsrc/wip/rust175/work/rust-bootstrap/lib" "/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc" "-vV" "-Wrust_2018_idioms" "-Wunused_lifetimes" "-Wsemicolon_in_expressions_from_macros" "-Dwarnings" "-Zunstable-options" "--check-cfg=values(bootstrap)" "-Z" "force-unstable-if-unmarked"
-------------

--- stderr
/usr/pkgsrc/wip/rust175/work/rustc-1.75.0-src/build/x86_64-unknown-netbsd/stage1/lib/librustc_driver-d78e0abcd3c75aa5.so: Shared object "libLLVM-16.so" not found

Building my own "bootstrap kit" for 1.74.1 using our pkgsrc setup produces a bootstrap kit which succeeds in building rust 1.75.0.

Looking at the build log above it is evident that the rust-supplied 1.74.1 bootstrap kit is built against an external LLVM, not the internal LLVM embedded in the rust distribution (ref. the version; the embedded LLVM is 17.x), and possibly that the rust bootstrap kit is not built with the --enable-cargo-native-static configure option(?) which according to my understanding minimizes or at least reduces the external dynamic dependencies. It is possible that these two conspire in contribution to this error.

Version it worked on

The bootstrap kit for x86_64-unknown-netbsd version 1.73.0 worked to build 1.74.1.

Version with regression

Trying to use the 1.74.1 bootstrap kit for x86_64-unknown-netbsd to build 1.75.0 fails, as above.

The following is for the working version that I built:

rustc --version --verbose:

: {1} setenv LD_LIBRARY_PATH /usr/pkgsrc/wip/rust175/work/rust-bootstrap/lib/
...
: {4} work/rust-bootstrap/bin/rustc --version --verbose
rustc 1.74.1 (a28077b28 2023-12-04) (built from a source tarball)
binary: rustc
commit-hash: a28077b28a02b92985b3a3faecf92813155f1ea1
commit-date: 2023-12-04
host: x86_64-unknown-netbsd
release: 1.74.1
LLVM version: 17.0.4
: {5}

This one does not have a dynamic dependency on LLVM:

: {5} ldd work/rust-bootstrap/bin/rustc
work/rust-bootstrap/bin/rustc:
        -lrustc_driver-23f6dd35cae7ab09 => /usr/pkgsrc/wip/rust175/work/rust-bootstrap/lib//librustc_driver-23f6dd35cae7ab09.so
        -lstd-00afc197bbba9b83 => /usr/pkgsrc/wip/rust175/work/rust-bootstrap/lib//libstd-00afc197bbba9b83.so
        -lpthread.1 => /usr/lib/libpthread.so.1
        -lc.12 => /usr/lib/libc.so.12
        -lgcc_s.1 => /usr/lib/libgcc_s.so.1
        -lexecinfo.0 => /usr/lib/libexecinfo.so.0
        -lelf.2 => /usr/lib/libelf.so.2
        -lm.0 => /usr/lib/libm.so.0
        -lz.1 => /usr/lib/libz.so.1
        -lstdc++.9 => /usr/lib/libstdc++.so.9
: {6} 

and

: {7} readelf -a /usr/pkgsrc/wip/rust175/work/rust-bootstrap/lib//librustc_driver-23f6dd35cae7ab09.so | grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libstd-00afc197bbba9b83.so]
 0x0000000000000001 (NEEDED)             Shared library: [libexecinfo.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.9]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.12]
: {8} 

(this is the version I've built using our pkgsrc setup and which can be used to build 1.75.0.)

Start with the stage1 build output and the rustc invocation from build/bootstrap, then inspect how the external LLVM dependency is recorded for the NetBSD bootstrap kit. Compare the failing 1.74.1 kit with the working locally built kit and its ldd/readelf results. Done means building Rust 1.75.0 reaches the stage1 library build without the missing libLLVM-16.so error.