NVIDIA / NVIDIA/OpenShell

feat(podman): relay callbacks through the rootless network namespace

Open
#2,540 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area:compute area:gateway state:review-ready state:stale
Dominant language
Rust
Stars
8.7k
Forks
1.3k
Avg merge
2d 11h
Merged PRs (30d)
253

Description

Problem Statement

Rootless Podman sandboxes using slirp4netns cannot connect directly to a
gateway callback listener in the host network namespace. Podman launches
slirp4netns with --disable-host-loopback; QEMU guest validation showed that
listeners bound to loopback, the host default-route address, and 0.0.0.0 were
all unreachable from an OpenShell-style user-defined Podman network.

PR #2492 therefore restricts direct local rootless callbacks to an explicitly
reported pasta helper. Explicit slirp4netns, missing helper metadata, and
unknown helpers fail closed unless the operator configures a genuinely remote
grpc_endpoint.

OpenShell should support local callbacks with rootless slirp4netns without
requiring operators to deploy remote gateway ingress.

Proposed Design

Add a driver-owned, byte-transparent callback relay spanning Podman's rootless
network namespace and the gateway host namespace:

sandbox supervisor
  -> host.openshell.internal:<callback-port>
  -> TCP relay inside `podman unshare --rootless-netns`
  -> mode-0600 Unix socket under $XDG_RUNTIME_DIR/openshell
  -> host-side relay
  -> gateway callback-only listener on IPv4 loopback

The initial proof in PR #2473 used two socat processes:

  1. A host-namespace Unix listener forwarding to the gateway loopback listener.
  2. A TCP listener launched with podman unshare --rootless-netns, forwarding
    to that Unix socket.

Production support should implement and supervise the relay in OpenShell code
rather than require socat:

  • Add a Podman-driver relay manager, or a narrowly scoped relay subcommand
    launched by the driver under podman unshare --rootless-netns.
  • Use one relay per driver/network and callback port, not one per sandbox.
  • Start and verify the relay before persisted sandboxes resume or new sandboxes
    are created.
  • Retry transient forwarding failures, surface terminal setup failures as
    driver preconditions, and clean up child processes and socket files on
    shutdown.
  • Report/request only an IPv4 loopback callback listener from the gateway. Do
    not broaden the primary listener.
  • Preserve supervisor-to-gateway mTLS end to end; the relay must not terminate
    TLS or inspect application payloads.
  • Create the Unix socket in a user-owned runtime directory with mode 0600,
    reject unsafe/stale socket paths, and avoid logging credentials or payloads.
  • Keep callback listeners restricted to sandbox-callable gRPC methods.
  • Probe podman unshare --rootless-netns support and fail closed when the relay
    topology cannot be established.

Explicitly reported slirp4netns should select this relay. Missing helper
metadata may use it only when the relay capability and reachability can be
verified; absence of rootlessNetworkCmd must not be treated as proof of a
legacy direct-listener topology.

Alternatives Considered

  • Bind the gateway to the host default-route address: works with pasta, but
    failed with slirp4netns because host loopback access is disabled.
  • Bind the gateway to 0.0.0.0: the diagnostic wildcard listener was also
    unreachable and would unnecessarily broaden the gateway attack surface.
  • Set host_gateway_ip: changing the alias target cannot bypass
    slirp4netns host-loopback isolation.
  • Require pasta permanently: safe as the current interim behavior, but
    excludes distributions and installations that use slirp4netns.
  • Require a remote grpc_endpoint: already supported and appropriate for
    deployments with secure routable ingress, but adds unnecessary
    infrastructure for a single-host local gateway.

Agent Investigation

  • Related contract: #2215.
  • Interim implementation: PR #2492.
  • Disposable QEMU tooling: PRs #2471 and #2473.
  • Fedora 44 / Podman 5.8.1 / pasta reached the host default-route listener.
  • CentOS Stream 10 / Podman 6.0.2 / pasta reached the host default-route
    listener.
  • Fedora 44 / Podman 5.8.1 / forced slirp4netns reported slirp4netns,
    launched it with --disable-host-loopback, and could reach neither the
    address-specific nor wildcard host listener.
  • Ubuntu 24.04 / Podman 4.9.3 omitted rootlessNetworkCmd; the live helper was
    slirp4netns with --disable-host-loopback, and neither host listener was
    reachable.
  • The PR #2473 guest relay made the callback path reachable, demonstrating the
    namespace-spanning relay topology, but intentionally masked the direct
    listener behavior measured above.
  • Duplicate searches for "rootless Podman relay", "slirp4netns callback", and
    "rootless network namespace" found no existing issue with this scope.

Acceptance Criteria

  • Rootless Podman with explicitly reported slirp4netns establishes local
    supervisor callbacks through the relay.
  • Relay startup is readiness-gated and failures prevent sandbox startup
    with an actionable error.
  • Supervisor-to-gateway mTLS and callback-only RPC authorization remain
    end to end.
  • Relay sockets and processes are securely created, supervised, restarted,
    and cleaned up.
  • Rootless pasta retains the direct default-route listener path.
  • Rootful Podman retains the exact bridge-gateway listener path.
  • Explicitly remote callback endpoints remain unchanged and request no
    local listener.
  • Missing helper metadata fails closed unless relay capability and
    reachability are positively verified.
  • QEMU E2E coverage includes Fedora and Ubuntu slirp4netns guests plus a
    pasta regression case.
  • Architecture, Podman networking, gateway configuration, and
    troubleshooting documentation describe the supported topologies.

Checklist

  • I've reviewed existing issues and the architecture docs
  • This is a design proposal, not a "please build this" request

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the related contract in #2215, interim implementation in PR #2492, and relay proof in PR #2473, then inspect the Podman driver entry point and podman unshare --rootless-netns behavior. Done means the acceptance criteria pass across slirp4netns, pasta, rootful Podman, remote endpoints, security checks, cleanup, and Fedora/Ubuntu QEMU coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, rust
Domain
infrastructure, networking, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.