deepseek-ai / deepseek-ai/DeepEP

Build fails with nvshmem installed as RPM in /usr/lib64 — unable to create wheel

Open
#383 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

### Description

When trying to build this package against `nvshmem` provided as an RPM (installed under `/usr/lib64` and `/usr/include/nvshmem_${CUDA_MAJOR_VERSION}`), the wheel build fails due to incorrect linking flags and library discovery issues.

Currently, the `setup.py` assumes local paths like `${nvshmem_dir}/lib` and `${nvshmem_dir}/include`. This breaks in RPM-based environments where `nvshmem` is installed system-wide.

#### Issues observed

1. **Link flags not generic**

* Existing `extra_link_args` used explicit `-l:libnvshmem_host.so` and `-l:libnvshmem_device.a`.
* These are brittle because they rely on filenames instead of sonames (`-lnvshmem_host`, `-lnvshmem_device`).
* Also, `nvshmem_bootstrap_uid.so` was incorrectly linked, but not actually required.

2. **RPATH handling**

* Previously only `${nvshmem_dir}/lib` was added to `-rpath`.
* On RPM-based installs, libraries are under `/usr/lib64/nvshmem/${CUDA_MAJOR_VERSION}` and `/usr/lib64`, requiring explicit rpath.

3. **Device linking**

* `nvcc_dlink` was missing system paths for `nvshmem_device`, causing unresolved references during device code linking.

4. **Wheel creation fails**

* Since the build cannot resolve `nvshmem_host` and `nvshmem_device` properly, `pip wheel .` fails to produce a wheel on systems where `nvshmem` is installed as an RPM.
* Error messages include missing `libnvshmem_host.so.3` and unresolved device symbols.

#### Changes needed

* Use **system include and library directories**:

```python
include_dirs.extend(['/usr/include', f'/usr/include/nvshmem_{os.getenv("CUDA_MAJOR_VERSION")}'])
library_dirs.extend(['/usr/lib64', f'/usr/lib64/nvshmem/{os.getenv("CUDA_MAJOR_VERSION")}'])
```
* Update **linker flags** to use sonames instead of filenames:

```python
extra_link_args.extend([
'-lnvshmem',
'-Wl,--no-as-needed',
'-lnvshmem_host',
'-lnvshmem_device',
f'-Wl,-rpath,/usr/lib64/nvshmem/{os.getenv("CUDA_MAJOR_VERSION")}:/usr/lib64'
])
```
* Update **device linking** with `nvcc`:

```python
nvcc_dlink.extend([
'-dlink',
'-L/usr/lib64',
f'-L/usr/lib64/nvshmem/{os.getenv("CUDA_MAJOR_VERSION")}',
'-lnvshmem_device'
])
```

#### Expected outcome

* Build succeeds when `nvshmem` is installed from RPM.
* Wheel (`.whl`) can be created and installed in a clean environment.
* Linker resolves `libnvshmem_host.so.3` and `libnvshmem_device` dynamically without hardcoding filenames.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.