Problem building 1.86.0 on Darwin due to problematic usage of DYLD_LIBRARY_PATH in bootstrap

Summary

Bootstrap is placing system directories such as /usr/local/lib into DYLD_LIBRARY_PATH, which allows stem name matching of libraries and can result in the incorrect version of system libraries being located during a lookup.

Here is the case I ran into:

Process:               cargo [54976]
Path:                  /usr/local/bin/cargo
Identifier:            cargo
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        ??? [54975]
Responsible:           cargo [54976]

Crashed Thread:        0

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Reason:    DYLD, [0x4] Symbol missing

Application Specific Information:
dyld: launch, loading dependent libraries
DYLD_LIBRARY_PATH=/tera/tera/debo/Projects/rustc/rustc-1.86.0-src/build/x86_64-apple-darwin/stage0-bootstrap-tools/x86_64-apple-darwin/release/deps:/usr/local/lib

Dyld Error Message:
  Symbol not found: __cg_jpeg_resync_to_restart
  Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /usr/local/lib/libJPEG.dylib
 in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO

Normally, the dynamic linker locates libraries using the absolute path recorded in an executable or another dynamic library, but setting DYLD_LIBRARY_PATH subverts that, because those directories are searched first using stem matching. This is intended to allow the use of special libraries, but it is not clear why bootstrap is placing a system library location (/usr/local/lib) into the path. In this case, the failure is due to the fact that the system ImageIO framework has its own copy of libjpeg, but putting /usr/local/lib in DYLD_LIBRARY_PATH causes /usr/local/lib/libjpeg.dylib to be located first using stem matching.

I was able to work around the problem by changing bootstrap to instead use DYLD_FALLBACK_LIBRARY_PATH and setting that to the appropriate search path in the environment before invoking the build. This works because DYLD_FALLBACK_LIBRARY_PATH is searched after the pathname searching is performed.

It is probably okay to place special build directories needed by bootstrap into DYLD_LIBRARY_PATH, but don't put any other locations there.

Command used

DESTDIR=/tmp/dst.rustc x.py install

Operating system

macOS 10.12.6

HEAD

Tarball build of 1.86.0:

05f9846f893b09a1be1fc8560e33fc3c815cfecb

Additional context

Vendoring sources to "/tera/tera/debo/Projects/rustc/rustc-1.86.0-src"
   Vendoring rust_team_data v1.0.0 (https://github.com/rust-lang/team#a5260e76) (/tera/tera/debo/.cargo/git/checkouts/team-679b79b2e58466cd/a5260e7/rust_team_data) to /tera/tera/debo/Projects/rustc/rustc-1.86.0-src/build/tmp/generate-copyright-vendor/rust_team_data-1.0.0
To use vendored sources, add this to your .cargo/config.toml for this project:

Building stage0 tool generate-copyright (x86_64-apple-darwin)
    Finished `release` profile [optimized] target(s) in 6.72s
Error: Failed get output from cargo-metadata: CargoMetadata { stderr: "dyld: Symbol not found: __cg_jpeg_resync_to_restart\n  Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO\n  Expected in: /usr/local/lib/libJPEG.dylib\n in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO\n" }

Caused by:
    `cargo metadata` exited with an error: dyld: Symbol not found: __cg_jpeg_resync_to_restart
      Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
      Expected in: /usr/local/lib/libJPEG.dylib
     in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
    
Command has failed. Rerun with -v to see more details.
Build completed unsuccessfully in 0:20:31

Start with the bootstrap environment setup used by DESTDIR=/tmp/dst.rustc x.py install, then trace where DYLD_LIBRARY_PATH is assembled and why /usr/local/lib is included. Reproduce the reported cargo metadata failure on Darwin; done means bootstrap no longer exposes system library directories through that variable and the build completes without the ImageIO/libJPEG lookup failure.