Find a way to avoid `LD_LIBRARY_PATH` when running tests

IIUC it is not possible to tell `cargo` to bake in the absolute path of a non-default library when compiling.

For example, one would do this in case of `gcc`:
https://stackoverflow.com/questions/8835108/how-to-specify-non-default-shared-library-path-in-gcc-linux-getting-error-whil (i.e. set `-rpath=...` to the correct value).

See here: https://github.com/rust-lang/cargo/issues/5077

I tried to modify the resulting binaries with `patchelf` but that somehow didn't work either, but I didn't really dig deep enough to understand why. Filing this bug so I don't forget.