DHI / DHI/mikecore-python

Bundled native libraries collide with same-named libs from a MIKE engine installation on LD_LIBRARY_PATH — import mikeio segfaults (proposal: auditwheel-style isolation)

Abierto
#45 0 comentarios 0 reacciones 1 asignado Reclamado por @JesperGr Ver en GitHub
Lenguaje dominante
Python
Estrellas
5
Forks
1
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

# Summary

`import mikeio` (and therefore `mikecore`) **segfaults in any Python process that has a MIKE installation's `bin` directory on `LD_LIBRARY_PATH`** (e.g. after sourcing `mikevars.sh`). This makes it impossible to use mikeio in the same environment — or the same process — as an installed MIKE engine.

The root cause is in how the `mikecore` Linux wheel bundles its native libraries, and it is fixable entirely inside the wheel build with standard tooling (`auditwheel` / `patchelf`). We have verified a fix experimentally: with a patched mikecore, mikeio and a running MIKE engine coexist happily **in a single process**.

# The problem, at a high level

The mikecore wheel ships its own copies of DHI and Intel native libraries (`libeum.so`, `libufs.so`, `libpfs2004.so`, `libMzCart.so`, `libimf.so`, `libiomp5.so`, `libsvml.so`, `libirng.so`, `libintlc.so.5`, …), built around 2015. A MIKE engine installation ships **newer libraries with exactly the same names** (2025 builds in current releases).

Because the two library sets share names, the dynamic linker mixes them, in two independent ways:

1. **`LD_LIBRARY_PATH` hijack.** mikecore's bundled libs carry `RUNPATH=$ORIGIN`. On Linux, `RUNPATH` is searched **after** `LD_LIBRARY_PATH` — so once a MIKE `bin` dir is on `LD_LIBRARY_PATH`, mikecore's 2015 `libeum.so` gets its dependencies (`libimf`, `libiomp5`, `libpfs2004`, …) resolved to the engine's 2025 copies. Result: `import mikeio` → SIGSEGV, even if no engine library was ever loaded by Python.
2. **soname dedup.** Even with (1) fixed, glibc loads at most one library per soname. Whichever `libimf.so` / `libiomp5.so` / … loads first is silently reused by the other party. With the engine loaded first, mikecore aborts with `free(): invalid size`; with mikecore first, the engine fails with e.g. `libpetsc.so.3.22: undefined symbol: __svml_ldexp2`.

Nothing about this is specific to one MIKE release — **any** MIKE version that ships same-named libraries will collide with mikecore's bundle. Conversely, once the bundle is properly isolated, mikecore is compatible with **every** MIKE version, past and future, because its libraries no longer share names with anything outside the wheel.

# What mikecore needs, concretely

Make the bundled native libraries fully self-contained and collision-free — exactly what `auditwheel repair` does for every manylinux wheel on PyPI:

1. **`RPATH` instead of `RUNPATH`.** Set `DT_RPATH=$ORIGIN` on every bundled `.so` (`patchelf --force-rpath --set-rpath '$ORIGIN'`). `RPATH` is searched **before** `LD_LIBRARY_PATH`, so the bundle always resolves to itself regardless of the caller's environment.
2. **Unique sonames.** Give every bundled library a name that cannot collide with a MIKE installation — e.g. a version/hash suffix (`libeum-mc2015.so`, or auditwheel's `libeum-a1b2c3d4.so` style): rename the files, `patchelf --set-soname` accordingly, and `patchelf --replace-needed old new` in every bundled lib that depends on them. The three libraries mikecore's Python code dlopens by absolute path (`libeum.so`, `libufs.so`, `libMzCart.so`) can keep their filenames — only their *sonames* and *NEEDED entries* need the treatment.

The simplest implementation is to run `auditwheel repair` as part of the Linux wheel build — it performs both steps (and the manylinux compliance check) automatically. If the bundle layout makes auditwheel awkward, the equivalent manual `patchelf` recipe is ~20 lines; we have a working script and can share it.

**Windows** likely deserves an audit too (DLL search order / same-basename shadowing when a MIKE `bin` is on `PATH`), but we have only verified the Linux side; the Windows failure mode — if any — may differ.

# Evidence

All experiments on Linux (WSL2, Ubuntu), mikecore from PyPI, a current MIKE engine installation, `mikevars.sh` + oneAPI `setvars.sh` sourced:

| Scenario | Stock mikecore | Patched mikecore (RPATH + unique sonames) |
|---|---|---|
| `import mikeio` + read dfsu | **SIGSEGV** (exit 139) | ✅ works |
| mikeio first, then load an engine library | **SIGSEGV** | ✅ works |
| engine library first, then mikeio | **abort**, `free(): invalid size` | ⚠️ still fails¹ |
| engine simulation + mikeio reading its output mid-run, one process | impossible | ✅ **works** — mikeio values match the engine's in-memory state on all wet elements |

¹ This residual ordering constraint is on the *caller's* side, not mikecore's: loading an engine library with `RTLD_GLOBAL` puts its symbols in the global scope where they override anything loaded later. Importing mikeio before the engine avoids it.

# Ask

Could the mikecore Linux wheel be built (or post-processed) with auditwheel-style isolation as described above? Happy to share the `patchelf` script, the test matrix, and a live demo, or to help test a candidate wheel against MIKE engine installations.

🤖 Drafted with [Claude Code](https://claude.com/claude-code)

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.