NVIDIA / NVIDIA/OpenShell

feat(sandbox): make privileged supervisor independent of workload-image code

Open
#2,750 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

arch:amd64 arch:arm64 area:compute area:sandbox area:supervisor os:linux state:stale test:e2e test:e2e-kubernetes topic:compatibility topic:networking topic:security
Dominant language
Rust
Stars
8.7k
Forks
1.3k
Avg merge
2d 11h
Merged PRs (30d)
253

Description

Problem Statement

The openshell-sandbox binary is supplied by OpenShell, but the privileged supervisor still runs tools from the workload image, including ip, nsenter, nft, and dmesg.

For example, consider an untrusted workload image that contains its own /usr/sbin/ip. During startup, the supervisor runs as root with SYS_ADMIN and NET_ADMIN and executes that path to create the sandbox network namespace. The path is absolute, so a PATH override cannot redirect it—but the file itself still belongs to the untrusted image. It could be a malicious binary or script, and a dynamically linked version also loads its interpreter and libraries from the image.

That code runs before the workload is sandboxed and can act with the supervisor's privileges. Even a non-malicious image can break startup by shipping an incompatible or missing helper.

The same problem exists for nsenter and nft. dmesg is worse because it is still resolved through PATH.

OpenShell should support arbitrary, untrusted workload images without executing any of their code as part of privileged supervisor setup.

Proposed Design

Establish this invariant:

A process retaining supervisor privileges must never execute or dynamically load code from the workload image. Workload entrypoints and SSH shells may execute only after privilege drop and sandbox hardening.

Deliver it in three phases:

  1. Remove the immediate workload-image dependency

    • Supply all privileged helpers from a trusted OpenShell image/location.
    • Replace Podman's workload-image shell health check.
    • Replace Kubernetes PVC seeding that currently runs workload-image sh as UID 0.
    • Add a hostile-image test with fake ip, nsenter, nft, dmesg, loaders, and libraries.
  2. Remove helper processes

    • Replace ip and nsenter with direct namespace syscalls and route netlink.
    • Replace nft with direct nf_tables netlink operations.
    • Replace dmesg bypass monitoring with NFLOG.
    • Use FD-owned network namespaces so /run/netns is no longer required.
  3. Enforce the boundary

    • Keep privileged networking/control separate from workload execution.
    • Route entrypoint and SSH execution through a permanently unprivileged process.
    • Deny execve/execveat in the privileged process after startup.

The certified self-contained artifact should be musl-static on Linux amd64 and arm64. Static glibc alone is insufficient because NSS/DNS paths may dynamically load modules.

Definition of Done

  • A hostile workload image cannot supply any executable, interpreter, loader, shared library, NSS module, or helper used by the privileged supervisor.
  • Docker, rootless Podman, Kubernetes combined/sidecar, and VM guest paths satisfy the same invariant.
  • Network namespace and nftables setup no longer spawn external helpers.
  • Bypass monitoring no longer uses dmesg or requires SYSLOG.
  • The privileged long-lived process cannot execute new programs after startup.
  • Workload entrypoints and SSH shells run only through an unprivileged execution path.
  • Hostile-image and cleanup tests cover normal startup, partial failure, shutdown, and crash recovery.

Why This Matters for Workspace Roots

The current OCI workdir guard forbids /bin, /sbin, /usr/bin, /usr/sbin, and library roots partly because the supervisor executes or loads code from them. Once this issue is complete, those paths no longer need to be reserved for supervisor safety. /proc, /sys, /dev, and OpenShell control paths remain protected for separate reasons.

Related Work

  • #1113 prevented simple PATH hijacking but did not make helper files independent of the workload image.
  • #1335 migrated rules to nftables but retained privileged CLI execution.
  • #2578 tracks reserved supervisor control namespaces and /run/netns cleanup.
  • #981 explores supervisor/workload separation on Kubernetes; this issue applies the executable trust boundary across all runtimes.

Agent Investigation

Audited at commit c174ad7f from origin/codex/2526-podman-oci-workdir.

Relevant code:

  • crates/openshell-supervisor-process/src/netns/mod.rs
  • crates/openshell-supervisor-process/src/bypass_monitor/mod.rs
  • crates/openshell-supervisor-process/src/process.rs
  • crates/openshell-supervisor-process/src/ssh.rs
  • crates/openshell-driver-podman/src/container.rs
  • crates/openshell-driver-kubernetes/src/driver.rs
  • crates/openshell-driver-vm/scripts/openshell-vm-sandbox-init.sh

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing privileged helper execution in crates/openshell-supervisor-process/src/netns/mod.rs, bypass_monitor/mod.rs, process.rs, and ssh.rs, then inspect the Podman, Kubernetes, and VM driver files listed in the issue. Map the existing startup and cleanup paths before attempting changes. Done means the stated trust-boundary invariant and all Definition of Done checks hold across the listed runtimes, with hostile-image coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, kubernetes, linux, rust
Domain
devops, networking, operating-systems, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.