openPMD / openPMD/openPMD-api

setup.py: More Control over Variants

Open
#492 1 comment 0 reactions 1 assignee Claimed by @ax3l View on GitHub
frontend: Python3 install MPI
Dominant language
C++
Stars
161
Forks
59
Avg merge
2d 22h
Merged PRs (30d)
7

Description

Our installs via `setup.py`, e.g. via PyPI, do not yet expose the same level of control over variants as our CMake (or spack) based installs.

Currently, all options are on `AUTO` detection which are often tricky with MPI.

In 0.8.0, we are adding `mpi4py` support which will expose a `pip install .[mpi]` "extra" in the `setup.py` install step. I am currently asking in `setuptools` (https://github.com/pypa/setuptools/issues/1712) how to query the extras during the build of an extension, so I can explicitly set `-DopenPMD_USE_MPI=ON/OFF` from it which will already relax a lot of issues.

One example that @ChristopherMayes saw: having a local MPI but only a serial HDF5, which will select MPI but cannot find a corresponding parallel HDF5. Due to upstream constrains in HDF5 and ADIOS1, we have to check MPI capabilities first in our configure logic. This `mpi` "extra" (variant) will make things easier since MPI is not on `AUTO` anymore.

Generally, we could think about controlling the defaults of our `option()`s in `CMakeLists.txt` via environment variables, so one can set them convieniently in all cases of wrapped scripting, e.g. via `pip install` from PyPI. (In such a scenario, no `configure` phase is available in the command line tools of python.)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.