`nvidia-fabricmanager-dev` ships the `libnvfm.so` symlink without `libnvfm.so.1`, so linking fails with `cannot find -lnvfm`

Summary

On Ubuntu 24.04, the nvidia-fabricmanager-dev package from the CUDA repository installs the linker symlink /usr/lib/x86_64-linux-gnu/libnvfm.so but not the libnvfm.so.1 library that symlink points at. -lnvfm cannot resolve, and the build stops at the linker.

This is the failure verbatim, with nvidia-fabricmanager-dev 580.178.04-1ubuntu1 installed:

/usr/bin/ld: cannot find -lnvfm: No such file or directory
collect2: error: ld returned 1 exit status

The headers ship correctly, so compilation succeeds and the failure appears only at the link step.

Environment

• Ubuntu 24.04, DGX OS 7.5.0, kernel 6.8.0-106-generic
• Package repository: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64
• Fabric Manager runtime installed: nvidia-fabricmanager 580.126.20-1

Reproduction

The package does not need to be installed — dpkg -c lists its contents:

apt-get download nvidia-fabricmanager-dev=610.57.04-1ubuntu1
dpkg -c nvidia-fabricmanager-dev_610.57.04-1ubuntu1_amd64.deb | grep libnvfm

The only match is the symlink:

./usr/lib/x86_64-linux-gnu/libnvfm.so -> libnvfm.so.1

There is no libnvfm.so.1 entry. Apart from that symlink the package contains only nv_fm_agent.h, nv_fm_types.h, changelog.Debian.gz and copyright.

Affected packages

I checked every nvidia-fabricmanager-dev package available in that repository — 11 in total — with dpkg -c.

Package version	Source package	libnvfm.so.1	Size (bytes)
nvidia-fabricmanager-dev-560 560.35.05-1	fabricmanager	present	1925144
nvidia-fabricmanager-dev-565 565.57.01-1	fabricmanager	present	2187512
nvidia-fabricmanager-dev-570 570.211.01-1	fabricmanager	present	2569880
nvidia-fabricmanager-dev-575 575.57.08-1	fabricmanager	present	2569880
nvidia-fabricmanager-dev-580 580.173.02-0ubuntu0.24.04.1	fabric-manager-580	present	2716368
nvidia-fabricmanager-dev 580.178.04-1ubuntu1	nvidia-fabricmanager	missing	—
nvidia-fabricmanager-dev 590.48.01-2ubuntu1	nvidia-fabricmanager	missing	—
nvidia-fabricmanager-dev 595.45.04-1ubuntu1	nvidia-fabricmanager	missing	—
nvidia-fabricmanager-dev 595.91.07-1ubuntu1	nvidia-fabricmanager	missing	—
nvidia-fabricmanager-dev 610.43.02-1ubuntu1	nvidia-fabricmanager	missing	—
nvidia-fabricmanager-dev 610.57.04-1ubuntu1	nvidia-fabricmanager	missing	—

Two things stand out.

The split follows the source package, not the version. Every package built from Source: nvidia-fabricmanager — the one whose binary package name carries no version suffix — is missing the library. Every package built from Source: fabricmanager or Source: fabric-manager-580 carries it. That pattern points at the packaging rules for nvidia-fabricmanager rather than at any single release.

The newest version is affected too. 610.57.04 is the most recent package in the repository and it is missing the library, so upgrading does not resolve this.

The runtime package does not supply the library either

The natural guess is that libnvfm.so.1 belongs to the runtime package and the dev package only adds the symlink on top. That is not the case. The runtime package ships no shared library at all:

$ dpkg -l | grep -i fabricmanager
nvidia-fabricmanager  580.126.20-1

$ dpkg -L nvidia-fabricmanager | grep -E "\.so|lib"
/usr/lib
/usr/lib/systemd
/usr/lib/systemd/system
/usr/lib/systemd/system/nvidia-fabricmanager.service

Nor does the daemon export the API for anyone to link against:

$ nm -D /usr/bin/nv-fabricmanager | grep -E " T fm(Activate|Deactivate|GetSupported|Connect|LibInit)"
$

So on an affected system there is no copy of libnvfm anywhere. The dev package is the only place it is meant to come from, and that is the package omitting it.

The dangling symlink makes diagnosis harder

The affected packages still install ./usr/lib/x86_64-linux-gnu/libnvfm.so -> libnvfm.so.1. After installation the system holds a symlink whose target does not exist. ls shows a libnvfm.so in the expected place and dpkg -L nvidia-fabricmanager-dev lists it, so the package looks correctly installed and cannot find -lnvfm reads like a mistake in the build rather than a gap in the package. Only ls -L, test -e, or resolving the symlink by hand reveals that nothing is behind it.

Shipping neither file would be easier to diagnose than shipping the symlink alone.

Workaround

Unpack a version-suffixed dev package and link against it where it lies, without installing it:

apt-get download nvidia-fabricmanager-dev-575
dpkg -x nvidia-fabricmanager-dev-575_575.57.08-1_amd64.deb lib575/
gcc -I lib575/usr/include prog.c -o prog \
    -L lib575/usr/lib/x86_64-linux-gnu -lnvfm \
    -Wl,-rpath,$PWD/lib575/usr/lib/x86_64-linux-gnu

Do not copy libnvfm.so.1 into /usr/lib/x86_64-linux-gnu. That puts a library from one release ahead of the Fabric Manager the machine is actually running, for every program on the machine. Keeping it behind -rpath confines it to the one binary being built.

I have used exactly this against a running Fabric Manager 580.126.20: fmGetSupportedFabricPartitions and fmActivateFabricPartition both work with the 575 library.

Second-order cost: the workaround forces headers and library apart

fmConnectParams changed across these releases. The 575 header defines fmConnectParams_v1; the 580 header defines fmConnectParams_v2, which adds an addressType field.

fmpm.cpp in this repository sets that field:

connectParams.addressType = NV_FM_API_ADDR_TYPE_UNIX;   // fmpm.cpp:424
connectParams.addressType = NV_FM_API_ADDR_TYPE_INET;   // fmpm.cpp:430

The affected package does ship the 580-era headers, so for this tool the library is the only piece that has to come from somewhere else. But that is what the workaround amounts to: headers from one release, library from another. Anyone whose code straddles the two now has to guard that field. Being pushed into a header and library split is a cost of the packaging defect, not a property of the API.

Request

The fix belongs to whoever builds the nvidia-fabricmanager source package rather than to this repository, so this issue is largely a request to route it there.

Separately, the failure that brings people here is more opaque than it needs to be. I have opened a pull request that makes make report which file is missing instead of stopping at cannot find -lnvfm.

Start by reviewing the package contents with the provided dpkg -c command and inspect fmpm.cpp around lines 424 and 430. The packaging fix belongs in the nvidia-fabricmanager source package rather than this repository; the separate make diagnostic change is already described as having an open pull request.