openmpi 4.1.6/8 with ucx (1.18) broken

Steps to reproduce

spack install openmpi@:4 fabrics=cma,ucx

compilation is fine

Error message

when executing a mpi application one gets

symbol lookup error: /nvme0/prudhomm/Devel/feelpp.spack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholde/linux-x86_64_v3/ucx-1.18.0-emtjpi3lgzhj5kooqe4onxod5wjk62iv/lib/ucx/libuct_rocm.so.0: undefined symbol: hsa_init

ldd gives :

ldd  /nvme0/prudhomm/Devel/feelpp.spack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholde/linux-x86_64_v3/ucx-1.18.0-emtjpi3lgzhj5kooqe4onxod5wjk62iv/lib/ucx/libuct_rocm.so
        linux-vdso.so.1 (0x00007ffe715cb000)
        libuct.so.0 => /nvme0/prudhomm/Devel/feelpp.spack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholde/linux-x86_64_v3/ucx-1.18.0-emtjpi3lgzhj5kooqe4onxod5wjk62iv/lib/libuct.so.0 (0x00007f7f29f84000)
        libucs.so.0 => /nvme0/prudhomm/Devel/feelpp.spack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholde/linux-x86_64_v3/ucx-1.18.0-emtjpi3lgzhj5kooqe4onxod5wjk62iv/lib/libucs.so.0 (0x00007f7f29f0f000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7f29ca2000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7f29bb7000)
        libucm.so.0 => /nvme0/prudhomm/Devel/feelpp.spack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholde/linux-x86_64_v3/ucx-1.18.0-emtjpi3lgzhj5kooqe4onxod5wjk62iv/lib/libucm.so.0 (0x00007f7f29b97000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f7f29fe8000)

nm gives:

nm  /nvme0/prudhomm/Devel/feelpp.spack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholde/linux-x86_64_v3/ucx-1.18.0-emtjpi3lgzhj5kooqe4onxod5wjk62iv/lib/ucx/libuct_rocm.so | grep hsa
                 U hsa_agent_get_info
                 U hsa_amd_agent_iterate_memory_pools
                 U hsa_amd_agent_memory_pool_get_info
                 U hsa_amd_agents_allow_access
                 U hsa_amd_ipc_memory_attach
                 U hsa_amd_ipc_memory_create
                 U hsa_amd_ipc_memory_detach
                 U hsa_amd_memory_async_copy
                 U hsa_amd_memory_lock
                 U hsa_amd_memory_pool_allocate
                 U hsa_amd_memory_pool_free
                 U hsa_amd_memory_pool_get_info
                 U hsa_amd_memory_unlock
                 U hsa_amd_pointer_info
                 U hsa_init
                 U hsa_iterate_agents
                 U hsa_signal_create
                 U hsa_signal_destroy
                 U hsa_signal_load_scacquire
                 U hsa_signal_store_screlease
                 U hsa_signal_wait_scacquire
0000000000009110 t uct_rocm_hsa_agent_callback
0000000000009080 t uct_rocm_hsa_pool_callback

nm on libhsa-runtime gives

nm /nvme0/prudhomm/Devel/feelpp.spack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholde/linux-x86_64_v3/hsa-rocr-dev-6.1.2-ulk6r53varidw3hwri4yttx3bi42dkhz/lib/libhsa-runtime64.so | grep hsa_init

this makes openmpi with ucx broken

Information on your system

• Spack: 1.0.0.dev0 (97edcb5acc364c8c61936d86ffb63c84c1d56af0)
• Python: 3.10.12
• Platform: linux-ubuntu22.04-zen2

General information

• I have run spack debug report and reported the version of Spack/Python/Platform
• I have searched the issues of this repo and believe this is not a duplicate
• I have run the failing commands in debug mode and reported the output