DHI / DHI/mikecore-python

Bundled native libraries collide with same-named libs from a MIKE engine installation on LD_LIBRARY_PATH — import mikeio segfaults (proposal: auditwheel-style isolation)

Open
#45 0 comments 0 reactions 1 assignee Claimed by @JesperGr View on GitHub
Dominant language
Python
Stars
5
Forks
1
PR merge metrics
No merged PRs in 30d

Description

# Summary

`import mikeio` (and therefore `mikecore`) **segfaults in any Python process that has a MIKE installation's `bin` directory on `LD_LIBRARY_PATH`** (e.g. after sourcing `mikevars.sh`). This makes it impossible to use mikeio in the same environment — or the same process — as an installed MIKE engine.

The root cause is in how the `mikecore` Linux wheel bundles its native libraries, and it is fixable entirely inside the wheel build with standard tooling (`auditwheel` / `patchelf`). We have verified a fix experimentally: with a patched mikecore, mikeio and a running MIKE engine coexist happily **in a single process**.

# The problem, at a high level

The mikecore wheel ships its own copies of DHI and Intel native libraries (`libeum.so`, `libufs.so`, `libpfs2004.so`, `libMzCart.so`, `libimf.so`, `libiomp5.so`, `libsvml.so`, `libirng.so`, `libintlc.so.5`, …), built around 2015. A MIKE engine installation ships **newer libraries with exactly the same names** (2025 builds in current releases).

Because the two library sets share names, the dynamic linker mixes them, in two independent ways:

1. **`LD_LIBRARY_PATH` hijack.** mikecore's bundled libs carry `RUNPATH=$ORIGIN`. On Linux, `RUNPATH` is searched **after** `LD_LIBRARY_PATH` — so once a MIKE `bin` dir is on `LD_LIBRARY_PATH`, mikecore's 2015 `libeum.so` gets its dependencies (`libimf`, `libiomp5`, `libpfs2004`, …) resolved to the engine's 2025 copies. Result: `import mikeio` → SIGSEGV, even if no engine library was ever loaded by Python.
2. **soname dedup.** Even with (1) fixed, glibc loads at most one library per soname. Whichever `libimf.so` / `libiomp5.so` / … loads first is silently reused by the other party. With the engine loaded first, mikecore aborts with `free(): invalid size`; with mikecore first, the engine fails with e.g. `libpetsc.so.3.22: undefined symbol: __svml_ldexp2`.

Nothing about this is specific to one MIKE release — **any** MIKE version that ships same-named libraries will collide with mikecore's bundle. Conversely, once the bundle is properly isolated, mikecore is compatible with **every** MIKE version, past and future, because its libraries no longer share names with anything outside the wheel.

# What mikecore needs, concretely

Make the bundled native libraries fully self-contained and collision-free — exactly what `auditwheel repair` does for every manylinux wheel on PyPI:

1. **`RPATH` instead of `RUNPATH`.** Set `DT_RPATH=$ORIGIN` on every bundled `.so` (`patchelf --force-rpath --set-rpath '$ORIGIN'`). `RPATH` is searched **before** `LD_LIBRARY_PATH`, so the bundle always resolves to itself regardless of the caller's environment.
2. **Unique sonames.** Give every bundled library a name that cannot collide with a MIKE installation — e.g. a version/hash suffix (`libeum-mc2015.so`, or auditwheel's `libeum-a1b2c3d4.so` style): rename the files, `patchelf --set-soname` accordingly, and `patchelf --replace-needed old new` in every bundled lib that depends on them. The three libraries mikecore's Python code dlopens by absolute path (`libeum.so`, `libufs.so`, `libMzCart.so`) can keep their filenames — only their *sonames* and *NEEDED entries* need the treatment.

The simplest implementation is to run `auditwheel repair` as part of the Linux wheel build — it performs both steps (and the manylinux compliance check) automatically. If the bundle layout makes auditwheel awkward, the equivalent manual `patchelf` recipe is ~20 lines; we have a working script and can share it.

**Windows** likely deserves an audit too (DLL search order / same-basename shadowing when a MIKE `bin` is on `PATH`), but we have only verified the Linux side; the Windows failure mode — if any — may differ.

# Evidence

All experiments on Linux (WSL2, Ubuntu), mikecore from PyPI, a current MIKE engine installation, `mikevars.sh` + oneAPI `setvars.sh` sourced:

| Scenario | Stock mikecore | Patched mikecore (RPATH + unique sonames) |
|---|---|---|
| `import mikeio` + read dfsu | **SIGSEGV** (exit 139) | ✅ works |
| mikeio first, then load an engine library | **SIGSEGV** | ✅ works |
| engine library first, then mikeio | **abort**, `free(): invalid size` | ⚠️ still fails¹ |
| engine simulation + mikeio reading its output mid-run, one process | impossible | ✅ **works** — mikeio values match the engine's in-memory state on all wet elements |

¹ This residual ordering constraint is on the *caller's* side, not mikecore's: loading an engine library with `RTLD_GLOBAL` puts its symbols in the global scope where they override anything loaded later. Importing mikeio before the engine avoids it.

# Ask

Could the mikecore Linux wheel be built (or post-processed) with auditwheel-style isolation as described above? Happy to share the `patchelf` script, the test matrix, and a live demo, or to help test a candidate wheel against MIKE engine installations.

🤖 Drafted with [Claude Code](https://claude.com/claude-code)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.