NVIDIA / NVIDIA/Personal-AI-Router

[Feature]: Run a services-only PAIR node as a container or Kubernetes pod

Open
#32 1 comment 0 reactions 1 assignee View on GitHub

@sherief-nv is already working on this.

Since Sep 9, 2026.

Dominant language
Go
Stars
1.4k
Forks
250
Avg merge
23h 27m
Merged PRs (30d)
1

Description

Area

Terminal / headless operation

User problem

A workload running on a Kubernetes cluster (a web app, a cron job, an agent) wants to use PAIR's endpoint, but PAIR runs on desktops and there is no documented way to put a node on the cluster. The two obvious attempts fail: a container on the pod network discovers nothing, because mDNS never leaves the local link; and pointing the workload at a node elsewhere on the LAN returns 403 loopback-only, by design.

Desired outcome

A documented, supported way to run a services-only node as a container, and as a pod on Kubernetes, that joins an existing cluster and gives the cluster's workloads an OpenAI-compatible endpoint that routes to the nodes with engines. Concretely: a Dockerfile built from the release asset, a page in docs/ that explains host networking and the loopback relay, and reference manifests.

Alternatives considered
  • Issuing in-cluster workloads mTLS client certificates so they can talk to a node's TLS personality directly. Rejected: that hands out a cluster identity per consumer.
  • Running the node on the Kubernetes host outside Kubernetes (systemd). Works, but is invisible to the cluster's scheduling, policy and lifecycle, and needs a relay anyway.
Additional context

CONTRIBUTING.md asks for maintainer alignment before adding a distribution channel. I have a validated implementation (PAIR v0.1.1 on OpenShift 4.22 with OVN-Kubernetes, paired with a Linux GPU node and a macOS node, five-minute soak with no failed requests) and will open it as a documentation PR with the files under services/container/. Happy to adjust the base image, drop the Dockerfile and keep only the page, or port the relay to Go if that is what you would accept.

Related: #30 fixes the address a node on an OVN-Kubernetes host advertises, which this work uncovered.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.