llvm / llvm/llvm-project

[OpenMP] libomp fails to build with LIBOMP_USE_HWLOC=ON against hwloc 3.x

Open
#215,942 1 comment 0 reactions 0 assignees View on GitHub
build-problem openmp
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

## Summary

Building the OpenMP runtime with `-DLIBOMP_USE_HWLOC=ON` against hwloc 3.x fails to compile: libomp's topology/affinity code is written against the hwloc 2.x API. Reproduced on `main` at `5b3766a6e2c5ef725d0bf9a2be84ff5b8fa84976` against hwloc `3.0.0a1-git` (`HWLOC_API_VERSION 0x00030000`) — 10 errors, all in `openmp/runtime/src/kmp_affinity.cpp`.

The same two constructs are present on `release/23.x` and `release/22.x`.

## Reproducer

```console
$ cmake -G Ninja -S llvm-project/runtimes -B build \
-DLLVM_ENABLE_RUNTIMES=openmp \
-DCMAKE_BUILD_TYPE=Release \
-DLIBOMP_USE_HWLOC=ON -DLIBOMP_HWLOC_INSTALL_DIR=$HWLOC3
$ ninja -C build
```

Configuration succeeds — `Looking for hwloc.h - found`, `Looking for hwloc_topology_init in $HWLOC3/lib64/libhwloc.so - found`, and the generated `kmp_config.h` carries `#define LIBOMP_USE_HWLOC 1`. The failure is at compile time.

## Break 1 — libomp's own hwloc-1.x compatibility shim rewrites the modern enum names into names hwloc 3.x removed (8 of the 10 errors)

`openmp/runtime/src/kmp.h:88-93`:

```c
#ifndef HWLOC_OBJ_NUMANODE
#define HWLOC_OBJ_NUMANODE HWLOC_OBJ_NODE
#endif
#ifndef HWLOC_OBJ_PACKAGE
#define HWLOC_OBJ_PACKAGE HWLOC_OBJ_SOCKET
#endif
```

`HWLOC_OBJ_NUMANODE` and `HWLOC_OBJ_PACKAGE` are *enumerators* of `enum hwloc_obj_type_e`, not macros, so the preprocessor cannot see them and `#ifndef` is unconditionally true on every hwloc version. libomp therefore always rewrites the modern spellings into the hwloc-1.x ones. That is invisible on hwloc 2.x, where `HWLOC_OBJ_NODE` / `HWLOC_OBJ_SOCKET` survive as deprecated aliases and the pair resolves back through macro blue-painting. hwloc 3.x removed both aliases, so every use of the *correct* modern name now fails to compile:

| Site | Source text | Diagnostic |
|---|---|---|
| `kmp_affinity.cpp:1734` | `case HWLOC_OBJ_PACKAGE:` | `use of undeclared identifier 'HWLOC_OBJ_SOCKET'` |
| `kmp_affinity.cpp:1736` | `case HWLOC_OBJ_NUMANODE:` | `use of undeclared identifier 'HWLOC_OBJ_NODE'` |
| `kmp_affinity.cpp:1736`, `:1738` | — | follow-on `duplicate case value 'HWLOC_OBJ_CORE'` from typo recovery |
| `kmp_affinity.cpp:1825` | `hwloc_get_obj_by_type(tp, HWLOC_OBJ_PACKAGE, 0)` | `use of undeclared identifier 'HWLOC_OBJ_SOCKET'` |
| `kmp_affinity.cpp:1896`, `:1899`, `:1970`, `:1973` | `memory->type == HWLOC_OBJ_NUMANODE` | `use of undeclared identifier 'HWLOC_OBJ_NODE'` |

Worth stressing: the affinity sources themselves are already correct — they spell the modern names throughout. Only the shim is wrong.

## Break 2 — `hwloc_cpukinds_get_info` is called with the hwloc 2.x 7-argument signature

`kmp_affinity.cpp:1864`, inside the `#if HWLOC_API_VERSION >= 0x00020400` block opened at `:1844`. That test has no upper bound, so hwloc 3.x takes the branch:

```
kmp_affinity.cpp:1864:11: error: no matching function for call to 'hwloc_cpukinds_get_info'
hwloc/cpukinds.h:137:1: note: candidate function not viable: requires 6 arguments, but 7 were provided
```

hwloc 3.x collapsed the trailing `unsigned *nr_infos, struct hwloc_info_s **infos` pair into a single `struct hwloc_infos_s **infosp` (`hwloc/cpukinds.h:136-142`).

## Scope

Only `kmp_affinity.cpp` fails. The rest of the runtime builds clean against hwloc 3.x — including `kmp_alloc.cpp`'s `HWLOC_LOCATION_TYPE_CPUSET` / memattrs path, which I expected to break and does not. No `-ferror-limit` truncation occurred, so the 10 errors below are the complete set.

## Impact

Distributors moving their system hwloc to 3.x must either disable `LIBOMP_USE_HWLOC` — losing `OMP_PLACES=numa_domains` and NUMA-aware affinity — or vendor a parallel hwloc 2.x. We currently pin hwloc 2.11.2 statically into `libomp.so` (hidden visibility) while shipping hwloc 3.x (`libhwloc.so.0`) for everything else in the same image, a dual-version arrangement this issue would make unnecessary.

## Proposed fix

1. `kmp.h`: version-gate the compat shim rather than testing `#ifndef` on enumerators — `#if HWLOC_API_VERSION < 0x00020000` — so the modern names reach the compiler unmodified on hwloc >= 2.x. This alone fixes 8 of the 10 errors.
2. `kmp_affinity.cpp`: add an `#if HWLOC_API_VERSION >= 0x00030000` branch for the `hwloc_cpukinds_get_info` call that reads the info array out of `struct hwloc_infos_s`.

We are happy to submit the patch.

## Environment

- `llvm-project` `main` @ `5b3766a6e2c5ef725d0bf9a2be84ff5b8fa84976`
- hwloc `3.0.0a1-git`, `HWLOC_API_VERSION 0x00030000`, built from `open-mpi/hwloc` master. Current master HEAD `48bd05a39eb509d9663453148d8a8f15760fa199` carries the same 6-argument prototype and likewise has no `HWLOC_OBJ_SOCKET` / `HWLOC_OBJ_NODE` in `hwloc.h`, `hwloc/cpukinds.h`, or `hwloc/deprecated.h`
- `aarch64-unknown-linux-gnu`, clang/lld 24.0.0git (self-hosted), CMake 4.2.1 + Ninja
- Built with `--no-default-config` so the diagnostics below carry no local flag customization

Full compiler output (10 errors)

```
llvm-project/openmp/runtime/src/kmp_affinity.cpp:1734:8: error: use of undeclared identifier 'HWLOC_OBJ_SOCKET'; did you mean 'HWLOC_OBJ_CORE'?
1734 | case HWLOC_OBJ_PACKAGE:
| ^~~~~~~~~~~~~~~~~
| HWLOC_OBJ_CORE
llvm-project/openmp/runtime/src/kmp.h:92:27: note: expanded from macro 'HWLOC_OBJ_PACKAGE'
92 | #define HWLOC_OBJ_PACKAGE HWLOC_OBJ_SOCKET
| ^~~~~~~~~~~~~~~~
$HWLOC3/include/hwloc.h:234:3: note: 'HWLOC_OBJ_CORE' declared here
234 | HWLOC_OBJ_CORE, /**< \brief Core.
| ^
llvm-project/openmp/runtime/src/kmp_affinity.cpp:1736:8: error: use of undeclared identifier 'HWLOC_OBJ_NODE'; did you mean 'HWLOC_OBJ_CORE'?
1736 | case HWLOC_OBJ_NUMANODE:
| ^~~~~~~~~~~~~~~~~~
| HWLOC_OBJ_CORE
llvm-project/openmp/runtime/src/kmp.h:89:28: note: expanded from macro 'HWLOC_OBJ_NUMANODE'
89 | #define HWLOC_OBJ_NUMANODE HWLOC_OBJ_NODE
| ^~~~~~~~~~~~~~
$HWLOC3/include/hwloc.h:234:3: note: 'HWLOC_OBJ_CORE' declared here
234 | HWLOC_OBJ_CORE, /**< \brief Core.
| ^
llvm-project/openmp/runtime/src/kmp_affinity.cpp:1736:8: error: duplicate case value 'HWLOC_OBJ_CORE'
1736 | case HWLOC_OBJ_NUMANODE:
| ^
llvm-project/openmp/runtime/src/kmp.h:89:28: note: expanded from macro 'HWLOC_OBJ_NUMANODE'
89 | #define HWLOC_OBJ_NUMANODE HWLOC_OBJ_NODE
| ^
llvm-project/openmp/runtime/src/kmp_affinity.cpp:1734:8: note: previous case defined here
1734 | case HWLOC_OBJ_PACKAGE:
| ^
llvm-project/openmp/runtime/src/kmp.h:92:27: note: expanded from macro 'HWLOC_OBJ_PACKAGE'
92 | #define HWLOC_OBJ_PACKAGE HWLOC_OBJ_SOCKET
| ^
llvm-project/openmp/runtime/src/kmp_affinity.cpp:1738:8: error: duplicate case value 'HWLOC_OBJ_CORE'
1738 | case HWLOC_OBJ_CORE:
| ^
llvm-project/openmp/runtime/src/kmp_affinity.cpp:1736:8: note: previous case defined here
1736 | case HWLOC_OBJ_NUMANODE:
| ^
llvm-project/openmp/runtime/src/kmp.h:89:28: note: expanded from macro 'HWLOC_OBJ_NUMANODE'
89 | #define HWLOC_OBJ_NUMANODE HWLOC_OBJ_NODE
| ^
llvm-project/openmp/runtime/src/kmp_affinity.cpp:1825:47: error: use of undeclared identifier 'HWLOC_OBJ_SOCKET'; did you mean 'HWLOC_OBJ_CORE'?
1825 | hwloc_obj_t o = hwloc_get_obj_by_type(tp, HWLOC_OBJ_PACKAGE, 0);
| ^~~~~~~~~~~~~~~~~
| HWLOC_OBJ_CORE
llvm-project/openmp/runtime/src/kmp.h:92:27: note: expanded from macro 'HWLOC_OBJ_PACKAGE'
92 | #define HWLOC_OBJ_PACKAGE HWLOC_OBJ_SOCKET
| ^~~~~~~~~~~~~~~~
$HWLOC3/include/hwloc.h:234:3: note: 'HWLOC_OBJ_CORE' declared here
234 | HWLOC_OBJ_CORE, /**< \brief Core.
| ^
llvm-project/openmp/runtime/src/kmp_affinity.cpp:1864:11: error: no matching function for call to 'hwloc_cpukinds_get_info'
1864 | if (hwloc_cpukinds_get_info(tp, idx, cpukinds[idx].mask,
| ^~~~~~~~~~~~~~~~~~~~~~~
$HWLOC3/include/hwloc/cpukinds.h:137:1: note: candidate function not viable: requires 6 arguments, but 7 were provided
137 | hwloc_cpukinds_get_info(hwloc_topology_t topology,
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~
138 | unsigned kind_index,
| ~~~~~~~~~~~~~~~~~~~~
139 | hwloc_bitmap_t cpuset,
| ~~~~~~~~~~~~~~~~~~~~~~
140 | int *efficiency,
| ~~~~~~~~~~~~~~~~
141 | struct hwloc_infos_s **infosp,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
142 | unsigned long flags);
| ~~~~~~~~~~~~~~~~~~~
llvm-project/openmp/runtime/src/kmp_affinity.cpp:1896:29: error: use of undeclared identifier 'HWLOC_OBJ_NODE'; did you mean 'HWLOC_OBJ_CORE'?
1896 | if (memory->type == HWLOC_OBJ_NUMANODE)
| ^~~~~~~~~~~~~~~~~~
| HWLOC_OBJ_CORE
llvm-project/openmp/runtime/src/kmp.h:89:28: note: expanded from macro 'HWLOC_OBJ_NUMANODE'
89 | #define HWLOC_OBJ_NUMANODE HWLOC_OBJ_NODE
| ^~~~~~~~~~~~~~
$HWLOC3/include/hwloc.h:234:3: note: 'HWLOC_OBJ_CORE' declared here
234 | HWLOC_OBJ_CORE, /**< \brief Core.
| ^
llvm-project/openmp/runtime/src/kmp_affinity.cpp:1899:37: error: use of undeclared identifier 'HWLOC_OBJ_NODE'; did you mean 'HWLOC_OBJ_CORE'?
1899 | if (memory && memory->type == HWLOC_OBJ_NUMANODE) {
| ^~~~~~~~~~~~~~~~~~
| HWLOC_OBJ_CORE
llvm-project/openmp/runtime/src/kmp.h:89:28: note: expanded from macro 'HWLOC_OBJ_NUMANODE'
89 | #define HWLOC_OBJ_NUMANODE HWLOC_OBJ_NODE
| ^~~~~~~~~~~~~~
$HWLOC3/include/hwloc.h:234:3: note: 'HWLOC_OBJ_CORE' declared here
234 | HWLOC_OBJ_CORE, /**< \brief Core.
| ^
llvm-project/openmp/runtime/src/kmp_affinity.cpp:1970:31: error: use of undeclared identifier 'HWLOC_OBJ_NODE'; did you mean 'HWLOC_OBJ_CORE'?
1970 | if (memory->type == HWLOC_OBJ_NUMANODE)
| ^~~~~~~~~~~~~~~~~~
| HWLOC_OBJ_CORE
llvm-project/openmp/runtime/src/kmp.h:89:28: note: expanded from macro 'HWLOC_OBJ_NUMANODE'
89 | #define HWLOC_OBJ_NUMANODE HWLOC_OBJ_NODE
| ^~~~~~~~~~~~~~
$HWLOC3/include/hwloc.h:234:3: note: 'HWLOC_OBJ_CORE' declared here
234 | HWLOC_OBJ_CORE, /**< \brief Core.
| ^
llvm-project/openmp/runtime/src/kmp_affinity.cpp:1973:39: error: use of undeclared identifier 'HWLOC_OBJ_NODE'; did you mean 'HWLOC_OBJ_CORE'?
1973 | if (memory && memory->type == HWLOC_OBJ_NUMANODE) {
| ^~~~~~~~~~~~~~~~~~
| HWLOC_OBJ_CORE
llvm-project/openmp/runtime/src/kmp.h:89:28: note: expanded from macro 'HWLOC_OBJ_NUMANODE'
89 | #define HWLOC_OBJ_NUMANODE HWLOC_OBJ_NODE
| ^~~~~~~~~~~~~~
$HWLOC3/include/hwloc.h:234:3: note: 'HWLOC_OBJ_CORE' declared here
234 | HWLOC_OBJ_CORE, /**< \brief Core.
| ^
10 errors generated.
```

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with the provided CMake and Ninja command using hwloc 3.x, then read the compatibility shim in openmp/runtime/src/kmp.h and the affected call sites in openmp/runtime/src/kmp_affinity.cpp. Verify the changes against both hwloc 3.x and the stated hwloc 2.x configurations; done means the OpenMP runtime builds with LIBOMP_USE_HWLOC=ON without losing the existing affinity paths.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp
Domain
build-system, hpc
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.