inference-cli should support Podman as a container runtime (rootless + NVIDIA CDI)

Search before asking

• I have searched the Inference issues and found no similar feature requests.

Description

inference server start requires a Docker daemon. container_adapter.py calls docker.from_env() at startup, so the README quickstart (pip install inference-cli && inference server start --dev) fails on the first command on hosts where Podman is the container runtime. Reproduced on Fedora 44 with podman 5.8 and no dockerd:

docker.errors.DockerException: Error while fetching server API version:
('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

The usual workaround, the podman socket (DOCKER_HOST=unix:///run/user/<uid>/podman/podman.sock), lets the CLI connect but does not carry GPU access. start_inference_container() requests the GPU with docker.types.DeviceRequest(capabilities=[["gpu"]]), which the Podman compatibility API ignores silently. The container starts with no CUDA access and no error, and fails later at first model load. Rootless podman has no NVIDIA container-runtime hook to fall back on either; the supported mechanism there is a CDI spec (nvidia-ctk cdi generate → podman run --device nvidia.com/gpu=all).

Searching the repo for "podman" returns no matches in code, issues, or PRs.

Requested behaviour, in decreasing order of preference:

1. Runtime autodetection: if podman is present and dockerd is not, shell out to podman run/create, translating the existing launch spec (ports, volumes, env), with GPU handled via CDI when a spec exists in ${XDG_DATA_HOME}/containers/cdi/ or /etc/containers/cdi/.
2. Minimum: fail loudly. If a GPU image launches over the podman compatibility socket, detect that device_requests were not honoured, or simply that the runtime is podman, and raise instead of starting a server without GPU access.

Use case

Developers and self-hosters on Fedora, RHEL, CentOS Stream, and Fedora Atomic workstations, where podman is the default and Docker is not installed. I run the inference server GPU image on Fedora with rootless podman and CDI successfully. The CLI is the only part of the toolchain that cannot reach it. The current workaround is a hand-maintained compose.yml (see Additional), which also gives up inference server start --dev and its ENABLE_BUILDER / NOTEBOOK_ENABLED conveniences.

Additional

Verified working rootless-podman launch spec, i.e. what the CLI would need to emit:

• CDI: nvidia-ctk cdi generate --output=$XDG_DATA_HOME/containers/cdi/nvidia.yaml, then --device nvidia.com/gpu=all.
• The compose form (deploy.resources.reservations.devices) also works; podman-compose 1.6+ translates it to the CDI device.
• Environment parity with --dev: ENABLE_BUILDER=True, NOTEBOOK_ENABLED=True, ports 127.0.0.1:9001 and 127.0.0.1:9002.

Related: #1075 (docs vs. actual behaviour drift on ports). A separate issue will cover the undocumented ENABLE_BUILDER flag.

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Start with container_adapter.py and the start_inference_container() entry point, then reproduce inference server start --dev with rootless Podman. Trace the existing launch spec and GPU DeviceRequest handling, and compare it with the requested Podman CDI behavior. Done means Podman is supported with GPU access and equivalent dev settings, or unsupported GPU launches fail loudly.