conda-forge / conda-forge/python-feedstock

Native aarch64 rebuild breaks cross-aarch64 C++ extension linking downstream (LDCXXSHARED regression)

Abierto
#880 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Shell
Estrellas
47
Forks
112
Merge medio
2 d 9 h
PR fusionados (30 d)
18

Descripción

Downstream cross-aarch64 recipes with a C++ Python extension (e.g. numba-cuda, libnvshmem) started failing at link time after python-feedstock's aarch64 build was flipped from cross to native. The compiler stack and the recipe-side sed are both unchanged; what changed is what CPython's own `configure` step emits into the sysconfigdata.

## When it landed

The "go native for arm builds" change removed `linux_aarch64: linux_64` from `conda-forge.yml`, landing on 2026-06-11 to 2026-06-12 across every currently-live branch except `3.12`:

- `main`: [`babe1dd57b`](https://github.com/conda-forge/python-feedstock/commit/babe1dd57b) in #874
- `3.13`: [`5a74e3a0ab`](https://github.com/conda-forge/python-feedstock/commit/5a74e3a0ab) in #877
- `3.11`: [`e20ad42441`](https://github.com/conda-forge/python-feedstock/commit/e20ad42441) in #875
- `3.10`: [`5ddb2eb7c5`](https://github.com/conda-forge/python-feedstock/commit/5ddb2eb7c5) in #876

Each PR bundled the flip with an unrelated main change (version bump or openssl backport); the flip itself is a one-liner in `conda-forge.yml`.

## Symptom (downstream cross-aarch64 link step)

```
$PREFIX/compiler_compat/ld: $BUILD_PREFIX/bin/../libexec/gcc/aarch64-conda-linux-gnu/14.3.0/liblto_plugin.so: error loading plugin: ... cannot open shared object file: No such file or directory
```

Misleading error — the actual issue is the aarch64 `ld` shim in `$PREFIX/compiler_compat/` being handed an x86_64 LTO plugin path via `--plugin`.

## Bisect (numba-cuda-feedstock main, last-green vs first-fail)

- `gcc_impl_linux-aarch64` / `binutils_impl_linux-aarch64` / `ld_impl_linux-aarch64`: identical (`14.3.0=h77f067d_19` / `2.45.1=default_hc5fa074_102` / `2.45.1=default_h27e1c4c_102`)
- `python` (linux-aarch64): `3.10.20=h28be5d3_0_cpython` → `3.10.20=h4f76b5d_1_cpython`

Extracting `_sysconfigdata_aarch64_conda_linux_gnu.py` (the "raw configure" variant, saved *before* the recipe-side sed) from each package:

- `h28be5d3_0` (cross-built): `'CXX': 'aarch64-conda-linux-gnu-c++'`
- `h4f76b5d_1` (native-built): `'CXX': 'aarch64-conda-linux-gnu-c++ -pthread'`

`LDCXXSHARED = "$(CXX) -shared"`, so it picks up the same `-pthread` on native. Then [`recipe/build_base.sh#L453-L456`](https://github.com/conda-forge/python-feedstock/blob/888e71e2/recipe/build_base.sh#L453-L456):

```bash
sed -i.bak "s@-pthread@-pthread -B $PREFIX/compiler_compat@g" sysconfigfile
```

turns the default sysconfigdata's `LDCXXSHARED` into `'g++ -pthread -B $PREFIX/compiler_compat -shared'`. Setuptools uses `LDCXXSHARED` verbatim for C++ extension links, so `-B $PREFIX/compiler_compat` reaches the cross-toolchain link line and routes `ld` through the aarch64 shim. On a linux-64 build host driving an x86_64 GCC, the aarch64 `ld` can't handle the x86_64 LTO plugin GCC hands it via `--plugin`, and the link fails.

## Why C-only cross-builds haven't been failing the same way

`LDSHARED` has always had `-B $PREFIX/compiler_compat` in it too, but patch [`0019-Fix-LDSHARED-when-CC-is-overriden-on-Linux-too.patch`](https://github.com/conda-forge/python-feedstock/blob/888e71e2/recipe/patches/0019-Fix-LDSHARED-when-CC-is-overriden-on-Linux-too.patch) makes distutils' `customize_compiler` strip the sysconfig `CC` prefix off `LDSHARED` and replace it with `os.environ['CC']`. Since `LDSHARED` starts with exactly the sysconfig `CC` string (`'gcc -pthread -B $PREFIX/compiler_compat'`), the entire `-pthread -B compiler_compat` chunk is consumed along with `gcc`, leaving only `-shared -Wl,...` in the resulting command. No equivalent exists for `LDCXXSHARED`, so it's used as-is.

## Why 3.12 isn't affected

The `3.12` branch of this feedstock still has `linux_aarch64: linux_64` in `conda-forge.yml` — it wasn't included in the "go native" flip. Correspondingly, only `linux_aarch64_python3.12` passes in [numba-cuda-feedstock run 28638905979](https://github.com/conda-forge/numba-cuda-feedstock/actions/runs/28638905979); all other Python versions fail on aarch64.

## Confirmed affected feedstocks

- [libnvshmem-feedstock#35](https://github.com/conda-forge/libnvshmem-feedstock/pull/35) — hit 2026-06-12, worked around with `-fno-use-linker-plugin`
- [numba-cuda-feedstock#92](https://github.com/conda-forge/numba-cuda-feedstock/pull/92) — hit 2026-07-03, same workaround

## Suggested fixes

1. Extend patch `0019` to also strip the sysconfig `CXX` prefix off `LDCXXSHARED` — symmetric with the C side. Preserves the sed's intent for native/system-gcc users.
2. Or scope the sed itself away from `CXX`/`LDCXXSHARED` so `-pthread -B compiler_compat` never enters those vars.

Happy to test either against numba-cuda-feedstock's CI once a rebuild is available.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.