MetOffice / MetOffice/mule

Mule installation fails to use shumlib installed via CMake

Open
#30 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
5
Forks
8
Avg merge
4h 16m
Merged PRs (30d)
6

Description

Mule can be installed with additional features such as UM spiral-search, packing, data_conv, etc extensions. Some of these extensions are available via [shumlib](/MetOffice/shumlib) open source which can be compiled during mule installation (assuming shumlib is available precompiled).

Recently shumlib has been updated to use CMake as a modern approach to build standard libraries. However, it currently lacks some shared libraries and header files when installed via CMake and mule installation fails e.g., like:

```
[INFO] Building ppibm module...
running build_ext
building 'um_ppibm.um_ieee2ibm32' extension
creating build
creating build/temp.linux-x86_64-3.9
creating build/temp.linux-x86_64-3.9/lib
creating build/temp.linux-x86_64-3.9/lib/um_ppibm
gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/lib64/python3.9/site-packages/numpy/core/include -I/......./SHUMLIB/include -I/usr/include/python3.9 -c lib/um_ppibm/um_ieee2ibm32.c -o build/temp.linux-x86_64-3.9/lib/um_ppibm/um_ieee2ibm32.o
lib/um_ppibm/um_ieee2ibm32.c:37:10: fatal error: c_shum_data_conv_version.h: No such file or directory
37 | #include "c_shum_data_conv_version.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
```

further, the linker also looks for the shared libraries in `SHUMLIB_INSTALL_PREFIX/lib`

```
[INFO] Building ppibm module...
running build_ext
building 'um_ppibm.um_ieee2ibm32' extension
creating build
creating build/temp.linux-x86_64-3.9
creating build/temp.linux-x86_64-3.9/lib
creating build/temp.linux-x86_64-3.9/lib/um_ppibm
gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/lib64/python3.9/site-packages/numpy/core/include -I/..../SHUMLIB/include -I/usr/include/python3.9 -c lib/um_ppibm/um_ieee2ibm32.c -o build/temp.linux-x86_64-3.9/lib/um_ppibm/um_ieee2ibm32.o
creating build/lib.linux-x86_64-3.9
creating build/lib.linux-x86_64-3.9/um_ppibm
gcc -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g build/temp.linux-x86_64-3.9/lib/um_ppibm/um_ieee2ibm32.o -L/..../SHUMLIB/lib -L/usr/lib64 -Wl,--enable-new-dtags,-R/..../SHUMLIB/lib -lshum_string_conv -lshum_byteswap -lshum_data_conv -o build/lib.linux-x86_64-3.9/um_ppibm/um_ieee2ibm32.cpython-39-x86_64-linux-gnu.so
/usr/bin/ld: cannot find -lshum_data_conv
collect2: error: ld returned 1 exit status
error: command '/usr/bin/gcc' failed with exit code 1
```
... and so on.

- related to MetOffice/shumlib/pull/35

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing Mule's installation and extension-build configuration for the SHUMLIB_INSTALL_PREFIX include and lib paths. Reproduce the failure with a CMake-installed shumlib, then verify that the required header and shared libraries are found while building the affected extensions. Done means Mule installs successfully with the shumlib features enabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, cmake, python
Domain
build-system, devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.