bazel-contrib / bazel-contrib/rules_perl

Support bazel-central-registry's rules_perl on `nixos`

Open
#80 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Starlark
Stars
31
Forks
49
PR merge metrics
No merged PRs in 30d

Description

Hello rules_perl owner,

I have a question about `rule_perl` .
(I did post this question under https://github.com/bazelbuild/bazel-central-registry/pull/3565 too)
We are using bazel to compile cc under `nix` system.
However, the `rule_perl` is broken.

You can reproduce it with a nix docker container.

## 1. Launch a nix docker container on Linux

```sh

mkdir /tmp/example; cd /tmp/example

## 1.1. Create a nix config.
## I installed the `pkgs.libcrypt` at first.
## However, it only has `libcrypt.so.2`, but `rule_perl` built perl wants `libcrypto.so.1`. Only `libxcrypt-legacy` includes it.
## See https://search.nixos.org/packages?channel=unstable&show=libxcrypt-legacy&from=0&size=50&sort=relevance&type=packages&query=libxcrypt

## Here, I also ask nix to install perl by itself. So, we can compare nix install perl vs rule_perl.

cat < default.nix
{ pkgs ? import {} }:
pkgs.mkShell {
buildInputs = [
pkgs.vim
pkgs.bazel_7
pkgs.libxcrypt-legacy
pkgs.perl
];
}
EOF

## 1.2. create an example
cat < MODULE.bazel
bazel_dep(name = "rules_perl", version = "0.2.5")
bazel_dep(name = "openssl", version = "3.3.1.bcr.1")
EOF

## 1.3. Start a nix docker container

sudo docker run --rm \
-v /tmp/example:/tmp/example \
--name my-nix-container -it nixos/nix bash
```

# 2. Within the nix, bazel build

```sh
cd /tmp/example
nix-channel --update
nix-shell
bazel build --sandbox_debug @openssl//:all
```

It fails with this error

```
/nix/store/fd118hwh7d1ncib4mdw56ylv3g9k0iyj-bash-5.2p37/bin/bash: line 3: external/rules_perl~~perl_repositories~perl_linux_amd64/bin/perl: cannot execute: required file not found
```

# 3. Debug

3.1. Run perl directly. It fails too.

```sh
## Path is copied from `bazel build --sandbox_debug`'s output
/root/.cache/bazel/_bazel_root/89a35363ec8de7131a16c2ed7419999a/sandbox/processwrapper-sandbox/48/execroot/_main/external/rules_perl~~perl_repositories~perl_linux_amd64/bin/perl
```

`ldd` the perl, `libcrypt.so.1` is not found.

```sh
ldd /root/.cache/bazel/_bazel_root/89a35363ec8de7131a16c2ed7419999a/sandbox/processwrapper-sandbox/48/execroot/_main/external/rules_perl~~perl_repositories~perl_linux_amd64/bin/perl
linux-vdso.so.1 (0x00007f508cbdf000)
libpthread.so.0 => /nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib/libpthread.so.0 (0x00007f508cbd4000)
libdl.so.2 => /nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib/libdl.so.2 (0x00007f508cbcf000)
libm.so.6 => /nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib/libm.so.6 (0x00007f508cae6000)
libcrypt.so.1 => not found
libutil.so.1 => /nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib/libutil.so.1 (0x00007f508cadf000)
libc.so.6 => /nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib/libc.so.6 (0x00007f508c800000)
/lib64/ld-linux-x86-64.so.2 => /nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib64/ld-linux-x86-64.so.2 (0x00007f508cbe1000)
```

3.2. set `LD_LIBRARY_PATH` to include `libcrypt.so.1`

```sh
ls /nix/store/*libxcrypt*/lib
```

Output

```
/nix/store/d550hx0zzirlbww967k0d0xs8asrb3ia-libxcrypt-4.4.38/lib:
libcrypt.la libcrypt.so libcrypt.so.2 libcrypt.so.2.0.0 pkgconfig

/nix/store/flvyd9qpx5gk6pr9l7ki0s841cx717ny-libxcrypt-4.4.36/lib:
libcrypt.la libcrypt.so libcrypt.so.2 libcrypt.so.2.0.0 pkgconfig

/nix/store/hcsh4083lvy8sim1ry04r83czfz1r1h6-libxcrypt-4.4.38/lib:
libcrypt.la libcrypt.so libcrypt.so.1 libcrypt.so.1.1.0 libxcrypt.so pkgconfig
```

Only `/nix/store/hcsh4083lvy8sim1ry04r83czfz1r1h6-libxcrypt-4.4.38/lib` has libcrypt.so.1. Use it.

```sh
(export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/nix/store/hcsh4083lvy8sim1ry04r83czfz1r1h6-libxcrypt-4.4.38/lib; ldd /root/.cache/bazel/_bazel_root/89a35363ec8de7131a16c2ed7419999a/sandbox/processwrapper-sandbox/48/execroot/_main/external/rules_perl~~perl_repositories~perl_linux_amd64/bin/perl)
```

`ldd` output looks fine.

```
linux-vdso.so.1 (0x00007f5c64300000)
libpthread.so.0 => /nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib/libpthread.so.0 (0x00007f5c642f5000)
libdl.so.2 => /nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib/libdl.so.2 (0x00007f5c642f0000)
libm.so.6 => /nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib/libm.so.6 (0x00007f5c64207000)
libcrypt.so.1 => /nix/store/hcsh4083lvy8sim1ry04r83czfz1r1h6-libxcrypt-4.4.38/lib/libcrypt.so.1 (0x00007f5c641cb000)
libutil.so.1 => /nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib/libutil.so.1 (0x00007f5c641c4000)
libc.so.6 => /nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib/libc.so.6 (0x00007f5c63e00000)
/lib64/ld-linux-x86-64.so.2 => /nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib64/ld-linux-x86-64.so.2 (0x00007f5c64302000)
```

3.3. try `perl` again with `LD_LIBRARY_PATH`

However, perl still fail

```sh
(export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/nix/store/hcsh4083lvy8sim1ry04r83czfz1r1h6-libxcrypt-4.4.38/lib; /root/.cache/bazel/_bazel_root/89a35363ec8de7131a16c2ed7419999a/sandbox/processwrapper-sandbox/48/execroot/_main/external/rules_perl~~perl_repositories~perl_linux_amd64/bin/perl)
```

Output

```
bash: /root/.cache/bazel/_bazel_root/89a35363ec8de7131a16c2ed7419999a/sandbox/processwrapper-sandbox/48/execroot/_main/external/rules_perl~~perl_repositories~perl_linux_amd64/bin/perl: cannot execute: required file not found
```

## 4. Root cause

```sh
readelf -Wl /root/.cache/bazel/_bazel_root/2190073a42434474a8b7f2bcd3c924b6/sandbox/processwrapper-sandbox/48/execroot/_main/external/rules_perl~~perl_repositories~perl_linux_amd64/bin/perl | grep "Requesting program interpreter"
```

Output
```
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
```

It should be related to the path of the `interpreter`.
It is supposed to be `/nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib64/ld-linux-x86-64.so.2`.

Explicitly add the `/nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib64/ld-linux-x86-64.so.2` in front of `perl`, works

```sh
/nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib64/ld-linux-x86-64.so.2 /root/.cache/bazel/_bazel_root/2190073a42434474a8b7f2bcd3c924b6/sandbox/processwrapper-sandbox/48/execroot/_main/external/rules_perl~~perl_repositories~perl_linux_amd64/bin/perl -v
```

Output

```
This is perl 5, version 36, subversion 0 (v5.36.0) built for x86_64-linux
...
```

## 5. Do you have any idea how to fix it?

### A tricky workaround

Adding `/nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib64/ld-linux-x86-64.so.2` in front of `bazel build` command doesn't help for sure.

`patchelf ` changes the `GLIBC` path. It fixes the perl, but this option is too trivial.

```sh
patchelf --set-interpreter \
/nix/store/81mi7m3k3wsiz9rrrg636sx21psj20hc-glibc-2.40-66/lib64/ld-linux-x86-64.so.2 \
/root/.cache/bazel/_bazel_root/2190073a42434474a8b7f2bcd3c924b6/sandbox/processwrapper-sandbox/48/execroot/_main/external/rules_perl~~perl_repositories~perl_linux_amd64/bin/perl
```

I think the right way to make perl also load the correct `GLIBC` when compile.
I went through perl's [bazel-central-registry code](https://github.com/bazelbuild/bazel-central-registry/tree/main/modules/rules_perl/0.2.5), but I didn't figure out how to fix it.

Thanks a lot

Contributor guide

Open the contributing guide

Research direction

Start with the rules_perl/0.2.5 module in bazel-central-registry and reproduce the failure using the provided default.nix, MODULE.bazel, Nix container, and bazel build command. Compare the packaged Perl interpreter and its ELF interpreter path; done means rule_perl's Perl executable runs and the Bazel build succeeds on NixOS without manual patchelf or loader-path workarounds.

Written by the indexing model from the issue text.

Assessment

Tech stack
nixos, perl
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.