Split sysroot into runtime libs and compile-time bits

I have built this rust executable:

fn main() {
    println!("hello world");
}

like so:

$ rustc -Cprefer-dynamic -Crpath=yes foo.rs

It has this runpath:

~/lix » readelf -d foo

Dynamic section at offset 0xfd50 contains 30 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libstd-998872f6596bd1c9.so]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../../../nix/store/y4wsjfgpavqs0y371frl5vgpc50fyr3w-rustc-1.82.0/lib/rustlib/aarch64-unknown-linux-gnu/lib:/home/jade/lix/outputs/out/lib:/nix/store/y4wsjfgpavqs0y371frl5vgpc50fyr3w-rustc-1.82.0/lib/rustlib/aarch64-unknown-linux-gnu/lib:/nix/store/5a2rdj5i1338isqdvivsn6yk7hyw4r4i-glibc-2.40-36/lib:/nix/store/b4mrzszd04v235l3dvfj0xkjwy7
rg9dj-gcc-13.3.0-lib/lib:/nix/store/31lhx9gjb4d4rsy3l9vxc63482z6vb8v-gcc-13.3.0-libgcc/lib]
 0x000000000000000c (INIT)               0x998
 0x000000000000000d (FINI)               0xd98
 0x0000000000000019 (INIT_ARRAY)         0x1fd00
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x1fd08
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x2d0
 0x0000000000000005 (STRTAB)             0x428
 0x0000000000000006 (SYMTAB)             0x2f0
 0x000000000000000a (STRSZ)              757 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000003 (PLTGOT)             0x1ff70
 0x0000000000000002 (PLTRELSZ)           192 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x8d8
 0x0000000000000007 (RELA)               0x788
 0x0000000000000008 (RELASZ)             336 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000000000001e (FLAGS)              ORIGIN BIND_NOW
 0x000000006ffffffb (FLAGS_1)            Flags: NOW ORIGIN PIE
 0x000000006ffffffe (VERNEED)            0x738
 0x000000006fffffff (VERNEEDNUM)         2
 0x000000006ffffff0 (VERSYM)             0x71e
 0x000000006ffffff9 (RELACOUNT)          9
 0x0000000000000000 (NULL)               0x0

Of particular note here is the /nix/store/y4wsjfgpavqs0y371frl5vgpc50fyr3w-rustc-1.82.0/lib/rustlib/aarch64-unknown-linux-gnu/lib: this is a problem! rustc and its transitive dependencies are over 1GB in size, which means that executables distributed with dynamic libstd cannot be distributed if they link to the toolchain's copy of libstd.

However, it's pretty reasonable to actually use the toolchain's libraries at runtime, they just have to be able to be configured to go somewhere else.

See, from the above, these libs from gcc (libgcc_s.so, libasah.so, etc etc etc) which basically everything is linked to:

$ nix path-info -rSsh /nix/store/b4mrzszd04v235l3dvfj0xkjwy7rg9dj-gcc-13.3.0-lib/lib /nix/store/31lhx9gjb4d4rsy3l9vxc63482z6vb8v-gcc-13.3.0-libgcc/lib
/nix/store/8rywcj8s7gx9iy4hwipfz7nb87s9rib9-libunistring-1.2       1.8M    1.8M
/nix/store/04aq2w58qlqjvwamcljh1hahz744hlzd-libidn2-2.3.7        359.0K    2.2M
/nix/store/31lhx9gjb4d4rsy3l9vxc63482z6vb8v-gcc-13.3.0-libgcc    147.4K  147.4K
/nix/store/9zk5kpadlnzhz4aq319hykb6anf71g7x-xgcc-13.3.0-libgcc   147.4K  147.4K
/nix/store/5a2rdj5i1338isqdvivsn6yk7hyw4r4i-glibc-2.40-36         39.8M   42.1M
/nix/store/b4mrzszd04v235l3dvfj0xkjwy7rg9dj-gcc-13.3.0-lib         8.6M   50.9M

It needs to be possible to ship rustc's runtime libs like libstd-foo.so without shipping an entire rest of a sysroot and compiler along with them. We can hack around this at a packaging level by doing some truly horrible fixups to every rust build after the fact, but it's not really reasonable to do to every single package: the compiler needs to be able to look somewhere different for its libstd dynamic libs so it just works.

The way that this would probably look like is that rustc has a separate install directory of runtime libs which is set up at configure time in a similar way to the main install directory and then that's the paths it would give to the linker when linking.

Discussion on Zulip: https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/libstd.2C.20rpath.2C.20dylibs.20and.20packaging

Relevant (I believe!) code:

https://github.com/rust-lang/rust/blob/84ce2e129a5b6a5af2f2ea27af5ad907a1aac419/compiler/rustc_session/src/filesearch.rs#L48-L59

Meta

rustc --version --verbose:

rustc 1.82.0 (f6e511eec 2024-10-15) (built from a source tarball)
binary: rustc
commit-hash: f6e511eec7342f59a25f7c0534f1dbea00d01b14
commit-date: 2024-10-15
host: aarch64-unknown-linux-gnu
release: 1.82.0
LLVM version: 18.1.8

Start with compiler/rustc_session/src/filesearch.rs at the referenced lines, then reproduce the issue using the provided rustc command and readelf output. Trace how rustc constructs the dynamic library search path. Done means runtime libraries can be installed separately while dynamically linked executables no longer reference the full compiler sysroot.