[Managed inference] Publish owned llama.cpp server image
- Dominant language
- TypeScript
- Stars
- 22.5k
- Forks
- 3.1k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 715
Description
## Summary
Add delivery slice 4 from accepted epic #8144: build and publish the NemoClaw-owned `llama-server` image used by the experimental managed llama.cpp path.
The image must be reproducible from the pinned upstream llama.cpp revision, published under `ghcr.io/nvidia/nemoclaw/llama-cpp-server`, and promoted only after both Linux architectures and their supply-chain evidence pass. This slice does not download a model, activate onboarding, or claim product support.
## Start condition
- #8173 / PR #8224 merged the explicit-only DGX Spark recipe and immutable implementation-spike tuple.
- The product and security boundary is accepted in #8144.
- NVIDIA's DGX Spark llama.cpp playbook remains the primary build and hardware reference; the pinned upstream llama.cpp source defines the server contract.
## Observable outcome
- Repository-owned container source builds `llama-server` from llama.cpp revision `22dc605c4ead20e36f447cc67b55ef87e523bd55` with CUDA enabled.
- Native `linux/amd64` and `linux/arm64` lanes publish immutable per-platform digests and one multiarchitecture OCI index.
- The published image is `ghcr.io/nvidia/nemoclaw/llama-cpp-server@sha256:...`, not an upstream or mutable tag.
- Publication emits and verifies an SPDX SBOM, SLSA provenance, signature, vulnerability result, exact source/base bindings, and anonymous exact-digest pull evidence.
- A protected DGX Spark lane proves that the arm64 artifact starts the pinned server binary and uses the NVIDIA GPU; CPU fallback is rejected.
- After publication succeeds, the production YAML recipe replaces the upstream implementation-spike image with the owned immutable index digest.
## Image contract
- Build from the pinned upstream source revision and a digest-pinned NVIDIA CUDA base.
- Build only the required `llama-server` runtime with CUDA, HTTPS/cURL support, and the DGX Spark GB10 architecture required by the accepted profile.
- Do not include model weights, model download credentials, mutable source resolution, UI assets, shell tools, or an operator-supplied launch command.
- Run the server as a dedicated non-root user. The final image must support a read-only root filesystem, a read-only model mount, and bounded writable temporary/CUDA-cache mounts supplied by the later lifecycle adapter.
- Preserve upstream license and notice material in the final image.
- Record OCI labels for the NemoClaw contract version, source repository and revision, CUDA base digest, platform, and publication revision.
## Workflow contract
- Pull requests build and inspect both native platforms without publishing packages or aliases.
- Pushes to `main` publish by digest first. No mutable consumer alias may move before both platform candidates and all required evidence pass.
- Assemble the multiarchitecture index from the exact validated platform descriptors.
- Verify anonymous exact-digest pulls and bind SBOM, provenance, signature, scan result, source revision, base digest, workflow run, and platform descriptors into a machine-readable publication receipt.
- Fail closed on a missing architecture, changed descriptor, mutable dependency, missing attestation, unsigned index, failed vulnerability policy, wrong binary identity, root runtime user, or unexpected listener/runtime surface.
- Reuse existing NemoClaw managed-image publication and evidence conventions where their contracts apply; do not couple the inference-server image to agent-image cohort promotion.
## Acceptance criteria
- Container and workflow contract tests cover immutable inputs, native platform selection, non-root runtime, exact binary/source labels, digest-only publication, multiarchitecture assembly, evidence binding, and fail-closed promotion.
- PR CI builds or validates both architectures without registry writes.
- The first `main` publication produces exact amd64, arm64, and index digests with SBOM, provenance, signature, vulnerability, and anonymous-pull evidence.
- Protected DGX Spark evidence identifies the exact image digest, source revision, host profile, GPU execution result, and server health result.
- The repository-owned YAML recipe pins the successful owned index digest; no TypeScript serving defaults are added.
- Existing vLLM automatic selection and the generic compatible-endpoint path remain unchanged.
## Non-goals
- No GGUF acquisition or shared model-cache ownership.
- No managed start, stop, restart, recovery, cleanup, or credential rotation.
- No onboarding option, automatic selection, provider registration, or documentation support claim.
- No model or agent qualification beyond the minimum artifact/GPU execution proof.
- No Podman, CPU-only, Metal, Vulkan, ROCm, Windows, multi-GPU, or distributed image contract.
Part of #8144.
Contributor guide
Assessment
This issue has not been assessed yet.