nvidia-container-cli not discoverable by gVisor/runsc when toolkit.installDir is non-default

Describe the bug

When toolkit.installDir is set to a non-standard location (required for OSes with a read-only /usr), nvidia-container-cli is installed to a path like /opt/nvidia/toolkit/nvidia-container-cli. This path is not on the system or containerd $PATH.

gVisor's runsc uses exec.LookPath("nvidia-container-cli") to locate the binary (source — see nvProxyPreGoferHostSetup), which fails with:

failed to locate nvidia-container-cli in PATH:
exec: "nvidia-container-cli": executable file not found in $PATH

This works with runc because nvidia-container-runtime resolves the binary path from its own config TOML. gVisor intentionally does not parse that config (source) and relies on standard PATH lookup instead.

To Reproduce

1. GPU Operator installs toolkit DaemonSet with toolkit.installDir set to a non-default path (e.g. /opt/nvidia/toolkit/).
2. gVisor/runsc is configured as an alternative runtime with nvproxy: true.
3. A GPU pod with runtimeClassName: gvisor fails to start because runsc calls exec.LookPath("nvidia-container-cli") during container creation, and the binary is not on the inherited PATH.

Expected behavior

nvidia-container-cli should be discoverable via standard PATH lookup after toolkit installation, regardless of installDir. Note that the motivation for using a non-default installDir is that /usr is read-only on these OSes, so symlinking into /usr/local/bin is not a viable solution.

Workaround

Extend containerd's PATH via a systemd drop-in:

# /etc/systemd/system/containerd.service.d/nvidia-toolkit.conf
[Service]
Environment=PATH=/opt/nvidia/toolkit:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Followed by systemctl daemon-reload && systemctl restart containerd.

Environment:

• nvidia-container-toolkit version: v1.17.4 (also reproduced with v1.19.1)
• NVIDIA Driver Version: 590.48.01 (open kernel modules)
• Host OS: Garden Linux 2150.4 (read-only /usr)
• Container Runtime Version: containerd 2.0.x
• CPU Architecture: x86_64
• GPU Model(s): T4
• Kubernetes: 1.31
• NVIDIA GPU Operator version: 24.9.2
• gVisor/runsc version: release-20260511.0

Start at the toolkit.installDir handling and the installation entry point that places nvidia-container-cli; compare the non-default-path behavior with runsc's nvProxyPreGoferHostSetup and exec.LookPath call. Done means a non-default installation is discoverable by standard PATH lookup for gVisor without writing to the read-only /usr, and the GPU pod startup scenario succeeds.