[Epic] Support llama.cpp as a managed local inference server

## Summary

Add `llama.cpp` (`llama-server`) as a first-class NemoClaw-managed local inference server.

Today an operator can start `llama-server` and select **Other OpenAI-compatible endpoint**. [Issue #2412](https://github.com/NVIDIA/NemoClaw/issues/2412) and [PR #5505](https://github.com/NVIDIA/NemoClaw/pull/5505) document that path. [Issue #7209](https://github.com/NVIDIA/NemoClaw/issues/7209) records validation through the generic path. That evidence does not establish first-class lifecycle or product support: NemoClaw does not identify, install, pin, configure, start, recover, diagnose, or qualify the server or its GGUF model.

This epic defines the product decision and delivery work required to add that ownership. The generic compatible-endpoint path remains available for self-managed or unsupported llama.cpp configurations.

## Implementation status

[Discussion #7636](https://github.com/NVIDIA/NemoClaw/discussions/7636) and this epic are accepted for experimental implementation. This is not a llama.cpp support activation.

- [x] Declarative serving catalog foundation: [#8151](https://github.com/NVIDIA/NemoClaw/issues/8151) / [PR #8152](https://github.com/NVIDIA/NemoClaw/pull/8152).
- [x] Cooperative existing-server attachment: [#8161](https://github.com/NVIDIA/NemoClaw/issues/8161) / [PR #8167](https://github.com/NVIDIA/NemoClaw/pull/8167).
- [x] Typed declarative llama.cpp catalog contract: [#8181](https://github.com/NVIDIA/NemoClaw/issues/8181) / [PR #8183](https://github.com/NVIDIA/NemoClaw/pull/8183) (merged).
- [x] Production `install-llama-cpp` YAML recipe and preset: [#8173](https://github.com/NVIDIA/NemoClaw/issues/8173) / [PR #8224](https://github.com/NVIDIA/NemoClaw/pull/8224) (merged).
- [ ] Managed artifact, acquisition, lifecycle, qualification, activation, and complete documentation.
- [x] Owned image contract: [#8231](https://github.com/NVIDIA/NemoClaw/issues/8231) / [PR #8235](https://github.com/NVIDIA/NemoClaw/pull/8235).
- [x] Trusted digest-first publication and supply-chain evidence contract: [#8250](https://github.com/NVIDIA/NemoClaw/issues/8250) / [PR #8252](https://github.com/NVIDIA/NemoClaw/pull/8252).
- [x] Dormant protected DGX Spark qualification lane: [#8260](https://github.com/NVIDIA/NemoClaw/issues/8260) / [PR #8266](https://github.com/NVIDIA/NemoClaw/pull/8266).
- [x] Activation-safe qualification test fixtures for the future YAML-only transition: [PR #8273](https://github.com/NVIDIA/NemoClaw/pull/8273) (merged).
- [ ] YAML-only DGX Spark activation, accepted physical evidence, owned-index publication, and production recipe digest pin.
- [x] Declarative GGUF identity and verification-plan foundation: [#8279](https://github.com/NVIDIA/NemoClaw/issues/8279) / [PR #8294](https://github.com/NVIDIA/NemoClaw/pull/8294).
- [x] Exact standard-cache GGUF acquisition, verification, and executor-only filesystem identity: [PR #8387](https://github.com/NVIDIA/NemoClaw/pull/8387) (merged).
- [x] Dormant Docker lifecycle binding and minimum provider-neutral authority seam: [#8395](https://github.com/NVIDIA/NemoClaw/issues/8395) / [PR #8418](https://github.com/NVIDIA/NemoClaw/pull/8418) (merged).
- [x] Crash-recoverable exact receipt publication: [#8414](https://github.com/NVIDIA/NemoClaw/issues/8414) / [PR #8422](https://github.com/NVIDIA/NemoClaw/pull/8422) (merged).
- [x] Experimental managed Docker onboarding, routing, diagnostics, recovery, and cleanup on DGX Spark: [#8433](https://github.com/NVIDIA/NemoClaw/issues/8433) / [PR #8448](https://github.com/NVIDIA/NemoClaw/pull/8448) (merged).
- [x] Declarative bounded llama.cpp protocol/model qualification foundation: [PR #8515](https://github.com/NVIDIA/NemoClaw/pull/8515) (merged; agent tuples remain unqualified).

Support activation remains a separate maintainer decision after the full managed artifact and compatibility evidence gates pass.

## Problem

Treating llama.cpp as an arbitrary compatible endpoint leaves these gaps:

- Onboarding cannot reliably fingerprint llama.cpp, vLLM, or another service on the same common port.
- Users must install and start the server, choose its flags, acquire a GGUF file, and keep the process running.
- NemoClaw has no immutable server or model identity to use for resume, recovery, diagnostics, or rollback.
- A responding endpoint does not establish model-template compatibility, structured tool calling, GPU execution, or agent compatibility.
- The server can fall back to CPU execution unless the launch and validation contract rejects that state.
- The server exposes UI and slot surfaces by default, and its official container binds broadly. Those defaults do not match NemoClaw's supported security boundary.
- Current status and doctor paths cannot separate server load, authentication, model identity, GPU offload, and sandbox-route failures.

## Desired behavior

An operator can attach to a cooperatively fingerprinted running llama.cpp server or select a NemoClaw-managed llama.cpp recipe. NemoClaw resolves and verifies the approved image and GGUF artifacts, starts the managed server on a private authenticated route, registers `llama-cpp-local`, and completes the qualified agent flow through `inference.local`.

Status, restart, recovery, rebuild, and restore retain the exact server, model, route, and ownership identities. A server or model outside the accepted matrix remains available only through `compatible-endpoint`.

## Product decision

[Discussion #7636](https://github.com/NVIDIA/NemoClaw/discussions/7636) and this epic are accepted as the implementation scope for an experimental managed llama.cpp provider. This authorizes the scoped implementation. It does not declare llama.cpp supported.

NemoClaw may advertise llama.cpp support only after the activation criteria pass and maintainers record the support decision. Before production artifact selection and support activation, maintainers must name:

- the product owner for the support matrix and activation decision;
- the engineering owner for provider, lifecycle, and agent compatibility;
- the OCI image and serving-catalog owner;
- the security and dependency-update owner;
- the llama.cpp update cadence and urgent-security-update process; and
- the compatibility, upgrade, rollback, and deprecation policy for pinned server/model recipes.

### Minimal v1 scope

| Dimension | v1 contract |
| --- | --- |
| Server | Upstream `llama-server`; no custom fork. |
| Deployment | One NemoClaw-owned container, one host, one allocated NVIDIA GPU, one model, one OpenShell gateway, one registered sandbox/trust principal, and one request slot. |
| Platforms | CUDA on `linux/amd64` and `linux/arm64`. Each activated host profile requires its own live E2E evidence. |
| Container runtime | Docker first. Podman activates only after the host-local inference contracts from [#7744](https://github.com/NVIDIA/NemoClaw/issues/7744) support the same lifecycle and evidence. |
| Models | Repository-owned recipes for exact GGUF file, revision, digest, quantization, context, chat template, license, and agent capabilities. |
| API | OpenAI Chat Completions only for agent traffic. Force `preferredInferenceApi=openai-completions` during selection and recovery. Use `/v1/models`, `/health`, `/props`, and `/metrics` only for bounded discovery and diagnostics. |
| Agents | Qualify OpenClaw, Hermes, and LangChain Deep Agents Code separately. One agent's tool-call result does not qualify another agent. |
| Support state | Keep the provider experimental until every claimed platform, model, and agent tuple passes the activation criteria. |

Use these initial identifiers:

- Interactive and noninteractive selection: `llama-cpp` for a cooperatively fingerprinted running server.
- Managed install selection: `install-llama-cpp`.
- OpenShell provider ID and durable provider value: `llama-cpp-local`.
- Display label: `Local llama.cpp`.
- Dedicated gateway credential: `NEMOCLAW_LLAMACPP_LOCAL_TOKEN`.
- Reserved v1 server port: `8081`.

Both first-class paths must use reserved port `8081`. They must fail on a conflict instead of moving silently. Other ports remain available through the generic compatible-endpoint path. The implementation must add `8081` to port validation, the `local-inference` policy, bundled-loopback handling, and route compatibility. It must treat `llama-cpp-local` as an endpoint-bearing provider.

## Architecture and ownership

### Serving catalog and readiness

The implementation must extend the managed serving design in [Discussion #7636](https://github.com/NVIDIA/NemoClaw/discussions/7636). It must not add another model registry, host-probe contract, or lifecycle format.

- Consume the versioned system-readiness report from [#7407](https://github.com/NVIDIA/NemoClaw/issues/7407).
- Use readiness facts for operating system, architecture, container runtime, NVIDIA GPU, available memory, driver compatibility, and storage admission.
- Put product selection in a llama.cpp serving preset and complete server/model behavior in a serving recipe.
- Compile only allowlisted launch settings. YAML must not contain shell fragments or arbitrary command templates.
- Record the resolved plan and immutable identities before the first managed side effect.
- Fail before acquisition or launch when required readiness is absent, unknown, stale, or incompatible.

If the catalog foundation from Discussion #7636 has not landed, the existing-server attachment slice may use the current provider modules. It must not create a temporary catalog or durable lifecycle contract that later needs a second migration.

Prerequisite status:

- [x] Versioned serving catalog/compiler: [#8151](https://github.com/NVIDIA/NemoClaw/issues/8151) / [PR #8152](https://github.com/NVIDIA/NemoClaw/pull/8152).
- [x] Versioned system-readiness schema: [#7407](https://github.com/NVIDIA/NemoClaw/issues/7407) / [PR #7544](https://github.com/NVIDIA/NemoClaw/pull/7544).
- [x] Provider-neutral host-local serving receipt: [PR #8060](https://github.com/NVIDIA/NemoClaw/pull/8060).
- [x] Minimum provider-neutral exact-engine, receipt, and journal seam folded into clean [PR #8418](https://github.com/NVIDIA/NemoClaw/pull/8418); broader PRs #8058 and #8069 are not dependencies of llama.cpp slice 6.

The typed schema/compiler work in [#8181](https://github.com/NVIDIA/NemoClaw/issues/8181) completed through [PR #8183](https://github.com/NVIDIA/NemoClaw/pull/8183). Production YAML completed through [#8173](https://github.com/NVIDIA/NemoClaw/issues/8173) / [PR #8224](https://github.com/NVIDIA/NemoClaw/pull/8224). The owned artifact is tracked by [#8231](https://github.com/NVIDIA/NemoClaw/issues/8231). Clean lifecycle delivery completed through [PR #8418](https://github.com/NVIDIA/NemoClaw/pull/8418) and [PR #8422](https://github.com/NVIDIA/NemoClaw/pull/8422); those PRs were not blocked on #8058 or #8069. Ownership, compatibility-policy, and update-cadence decisions remain activation gates. The existing-server attachment completed through [#8161](https://github.com/NVIDIA/NemoClaw/issues/8161) / [PR #8167](https://github.com/NVIDIA/NemoClaw/pull/8167) without claiming ownership of the server or model.

### Server and model artifacts

Each supported recipe must pin:

- llama.cpp source commit and immutable OCI image digest;
- CUDA base and supported driver boundary;
- GGUF repository, exact revision, exact filename, SHA-256 digest, expected size, quantization, and model license;
- logical model ID, chat template, context size, slot count, GPU/offload policy, and bounded request limits;
- supported agent and protocol capabilities;
- health, readiness, disabled idle sleep, graceful-stop, and recovery behavior.

The shipping image must be NemoClaw-owned. It must run as a non-root user with a read-only root filesystem and read-only model mount. It must provide bounded writable temporary storage for CUDA JIT cache and required temporary files. Publication must include multiarchitecture runtime tests, an SBOM, provenance, signing, and vulnerability scanning. A digest-pinned upstream image may be used only for an implementation spike.

Reuse the Hugging Face acquisition mechanism and standard cache already used by the DGX Spark vLLM install path. It must request the exact YAML-declared revision and file, then verify the canonical local file size and digest before use. Server startup must use that verified local file in offline mode. A gated-model credential may exist only for the shared acquisition operation and must not enter the llama.cpp image, server environment, logs, registry, snapshot, or sandbox.

### Routing and security

The managed server must be reachable only through the private host/gateway topology required by `https://inference.local/v1`.

- Require a dedicated provider credential and keep it separate from `OPENAI_API_KEY`.
- Do not expose an unauthenticated listener to the LAN or public network.
- Disable or gateway-block the Web UI, public slot inspection, router mode, Model Context Protocol (MCP) proxy, server-side `--tools`, server-side agent mode, multimodal projection, and model-initiated downloads.
- Pass a verified local `--model`, `--no-mmproj`, and offline mode. Reject non-text content before forwarding and deny server egress during runtime.
- Keep prompt and response bodies out of production logs.
- Apply request-body, token, time, concurrency, process, memory, and storage bounds outside or around `llama-server` as required.
- Pin `--sleep-idle-seconds -1` so `/health` cannot report success for an unloaded model. Treat health as load/readiness evidence only.
- Define a cooperative fingerprint from bounded server metadata and local runtime evidence. Treat it as selection evidence, not an authentication or security identity.
- Verify the non-path model alias, template capability, and GPU execution separately.

The two first-class paths have separate credential ownership:

- The managed path generates a key, persists it with mode `0600` in the gateway-scoped NemoClaw host state, injects it through a secret file, and records only a hash in the receipt. Rotation must update the server and OpenShell provider atomically.
- The existing-server path requires an operator-supplied native llama.cpp key through `NEMOCLAW_LLAMACPP_LOCAL_TOKEN`. A server without native key enforcement remains on the generic compatible-endpoint path.

### State and lifecycle

Persist one ownership receipt for the managed server. The receipt must include the serving plan digest, image digest, model digest, runtime identity, reserved port, provider identity, and lifecycle generation.

- `start` and `restart` must reuse the verified local artifact from the standard Hugging Face cache.
- `recover` must replace only a NemoClaw-owned missing or failed server.
- `rebuild`, snapshot/restore, and clone must preserve the provider/model recipe and rebind the OpenShell route without copying provider credentials into sandbox state.
- Snapshots must record model identity, not duplicate GGUF bytes.
- Restore must verify the exact file in the standard Hugging Face cache or rerun the shared acquisition effect.
- Uninstall must not remove files from the shared Hugging Face cache. It must not remove a foreign process, container, image, or model file.
- Deep Agents Code provider changes must retain its current re-onboard boundary. OpenClaw and Hermes may use `nemoclaw inference set` after their agent-specific validation passes.
- A second sandbox or a conflicting llama.cpp route on the same gateway must fail before effects. Multi-sandbox sharing requires a later accepted issue.

## Required delivery slices

- [x] **1. Accept the product contract and support matrix.** Discussion #7636 and the maintainer decisions recorded in this epic confirm the identifiers, experimental state, initial DGX Spark/GB10 host profile, NVIDIA Nemotron GGUF recipe, Chat Completions and agent scope, ownership boundary, and activation exit criteria. This remains an experimental implementation decision, not a support claim.
- [x] **2. Add cooperative server fingerprinting and existing-server attachment.** Distinguish llama.cpp, vLLM, and arbitrary OpenAI-compatible servers in supported deployments. Require port `8081`, a non-path alias, and an operator-supplied native key. Missing, conflicting, or ambiguous signals route to manual compatible-endpoint selection instead of a guessed local provider. Completed by [#8161](https://github.com/NVIDIA/NemoClaw/issues/8161) / [PR #8167](https://github.com/NVIDIA/NemoClaw/pull/8167).
- [x] **3. Add the serving catalog entries and compiler validation.** Define llama.cpp recipes and presets, exact artifact fields, agent capabilities, allowed launch settings, and diagnostics. Reuse Discussion #7636 contracts.
- [x] Declarative catalog/compiler foundation: [#8151](https://github.com/NVIDIA/NemoClaw/issues/8151) / [PR #8152](https://github.com/NVIDIA/NemoClaw/pull/8152).
- [x] Typed llama.cpp schema/compiler extension: [#8181](https://github.com/NVIDIA/NemoClaw/issues/8181) / [PR #8183](https://github.com/NVIDIA/NemoClaw/pull/8183) (merged).
- [x] Production repository-owned YAML recipe and preset: [#8173](https://github.com/NVIDIA/NemoClaw/issues/8173) / [PR #8224](https://github.com/NVIDIA/NemoClaw/pull/8224) (merged).
- [ ] **4. Publish the managed OCI artifact.** Tracked by [#8231](https://github.com/NVIDIA/NemoClaw/issues/8231). Select the owned registry and workflow. Build pinned `linux/amd64` and `linux/arm64` CUDA images, compile out optional UI assets where supported, runtime-disable and gateway-block unused surfaces, run as non-root, publish immutable evidence, and prove real GPU execution.
- [x] Owned, pinned, non-root multiarchitecture image contract and secretless native PR builds: [PR #8235](https://github.com/NVIDIA/NemoClaw/pull/8235).
- [x] Trusted digest-first publication, SBOM, provenance, signing, scan, and receipt contract: [#8250](https://github.com/NVIDIA/NemoClaw/issues/8250) / [PR #8252](https://github.com/NVIDIA/NemoClaw/pull/8252).
- [x] Dormant trusted-main DGX Spark qualification and evidence lane: [#8260](https://github.com/NVIDIA/NemoClaw/issues/8260) / [PR #8266](https://github.com/NVIDIA/NemoClaw/pull/8266).
- [ ] Provision and activate the protected Spark tuple through YAML, accept physical evidence, publish the validated owned index, and pin its digest into the repository-owned recipe.
- [x] **5. Reuse GGUF acquisition and verify local input.** Tracked by [#8279](https://github.com/NVIDIA/NemoClaw/issues/8279). Reuse the existing Hugging Face acquisition mechanism and standard cache. Select the exact YAML-declared file and revision, run storage preflight, isolate the credential, verify the canonical local file, and pass a typed verified artifact to llama.cpp.
- [x] YAML/schema/type model identity, deterministic verification-plan compiler, and local-entry verification foundation: [PR #8294](https://github.com/NVIDIA/NemoClaw/pull/8294).
- [x] Production host-local llama.cpp materializer consumed by protected DGX Spark qualification: [PR #8338](https://github.com/NVIDIA/NemoClaw/pull/8338) (merged).
- [x] Provider-neutral reuse of the existing Hugging Face acquisition effect: [PR #8371](https://github.com/NVIDIA/NemoClaw/pull/8371) (merged).
- [x] Exact standard-cache GGUF resolution, digest verification, and pre-materialization filesystem-identity revalidation: [PR #8387](https://github.com/NVIDIA/NemoClaw/pull/8387) (merged).
- [x] **6. Implement launch and lifecycle.** Clean dormant lifecycle binding completed through [#8395](https://github.com/NVIDIA/NemoClaw/issues/8395) / [PR #8418](https://github.com/NVIDIA/NemoClaw/pull/8418); crash-recoverable receipt publication completed through [#8414](https://github.com/NVIDIA/NemoClaw/issues/8414) / [PR #8422](https://github.com/NVIDIA/NemoClaw/pull/8422). The dormant controller now materializes the bounded command, allocates one GPU, rejects CPU fallback, persists exact ownership, and provides crash-safe lifecycle and receipt boundaries for activation work.
- [x] **7. Integrate onboarding, routing, and runtime mutation.** Add provider aliases, menus, interactive and noninteractive flows, model discovery, dedicated credentials, `inference.local` routing, endpoint-aware compatibility, network policy, status, doctor, recovery, and agent-specific `inference set` behavior. Force Chat Completions and skip the Responses probe. Existing-server onboarding landed in [PR #8167](https://github.com/NVIDIA/NemoClaw/pull/8167); experimental managed Docker onboarding, routing, status, doctor, recovery, cleanup, and provider-scoped lifecycle integration landed in [#8433](https://github.com/NVIDIA/NemoClaw/issues/8433) / [PR #8448](https://github.com/NVIDIA/NemoClaw/pull/8448).
- [x] **8. Qualify protocol and model behavior.** Test synchronous and streaming Chat Completions, cancellation, usage, structured output, function-call emission, tool-result continuation, context limits, authentication, malformed requests, and timeouts for each supported model/agent tuple. Maintainer-accepted for delivery sequencing through PRs #8515 and #8579; physical DGX Spark validation is deferred until hardware is available.
- [x] Declarative, bounded protocol/model qualification foundation: [PR #8515](https://github.com/NVIDIA/NemoClaw/pull/8515). YAML selects the exact probes and bounds; protected CI validates the runner and receipt contract.
- [x] Accept the OpenClaw tuple for delivery sequencing through `inference.local`; physical DGX Spark validation remains deferred and the current recipe intentionally remains `agents: []` until the separate support-activation decision.
- [ ] **9. Add security, observability, and protected E2E.** Verify disabled surfaces, request limits, log redaction, readiness, metrics, GPU telemetry, failure diagnostics, lifecycle recovery, and exact-head live E2E for every activated platform and container runtime. PR #8167 supplies attachment-path credential, probe, port, route-containment, and protected exact-head E2E evidence; managed image, lifecycle, telemetry, and activated-platform evidence remain.
- [x] **10. Publish user and operator documentation.** Compare llama.cpp with Ollama, vLLM, and NVIDIA NIM. Document existing-server and managed paths, shared Hugging Face cache use, model lifecycle, security boundary, diagnostics, upgrade, rollback, and the generic compatible-endpoint fallback. Existing-server command/reference documentation landed in PR #8167; complete managed-path and operator documentation landed in [PR #8610](https://github.com/NVIDIA/NemoClaw/pull/8610).

## Acceptance criteria

### Selection and routing

- Interactive onboarding offers a cooperatively fingerprinted running llama.cpp server and the managed install path when the readiness report permits them.
- `NEMOCLAW_PROVIDER=llama-cpp NEMOCLAW_MODEL= NEMOCLAW_LLAMACPP_LOCAL_TOKEN= nemoclaw onboard --non-interactive --yes-i-accept-third-party-software` attaches to a server that meets the fingerprint, alias, port, and authentication contract.
- `NEMOCLAW_PROVIDER=install-llama-cpp NEMOCLAW_LLAMACPP_RECIPE= nemoclaw onboard --non-interactive --yes-i-accept-third-party-software` materializes the exact managed recipe and persists its logical model alias as the runtime model ID.
- Both paths register `llama-cpp-local` with a dedicated credential and route agent traffic through `https://inference.local/v1`.
- Cooperative fingerprint tests do not misclassify llama.cpp as vLLM or vLLM as llama.cpp. An arbitrary, spoofed, or ambiguous server stays on the generic path.
- The managed server sets a stable, non-path model alias. An attached server must report such an alias in its inventory; otherwise it stays on the generic path.
- Provider compatibility includes the endpoint. A route that has the same model but a different port or server identity is incompatible.
- A second sandbox request on the same gateway fails before server, provider, route, or registry mutation.

### Artifact and execution evidence

- Status reports the immutable server and model identities, selected recipe, host profile, lifecycle state, load/readiness state, and route health without exposing a credential.
- A managed server starts without network access after its image and GGUF file are present.
- Runtime evidence proves that the selected process uses the allocated NVIDIA GPU. CPU-only fallback fails with an actionable diagnostic.
- Insufficient GPU memory, disk space, driver compatibility, checksum mismatch, model-template mismatch, and port conflict fail before registration or leave a complete rollback.
- Restart and recovery use the recorded image and model digests. They do not resolve a mutable upstream tag again.
- Recovery reads the gateway-scoped secret state. It does not depend on the original shell environment.

### Agent compatibility

- Every supported tuple records the agent, exact model, GGUF quantization, chat template, context, llama.cpp image digest, platform, and container runtime.
- Each tuple completes a normal response, streaming response, structured function call, tool-result continuation, and multi-turn agent flow through `inference.local`.
- The validation checks the emitted tool-call structure and the agent's execution of the intended tool. Server startup alone is not sufficient.
- Selection, recovery, and sandbox configuration set `openai-completions` and replace any retained Responses preference.
- The v1 path does not advertise OpenAI Responses API parity, embeddings, reranking, multimodal input, or parallel tool calls.
- Deep Agents Code re-onboards when its provider changes. Its llama.cpp qualification remains separate from OpenClaw and Hermes.

### Security and lifecycle

- No supported configuration exposes the server, health endpoints, model inventory, UI, or slot state to an untrusted network.
- The runtime is non-root, the model mount is read-only, and server-side filesystem or shell tools are disabled.
- Credentials and gated-model tokens do not appear in process arguments, logs, snapshots, registry output, or sandbox configuration.
- Ownership checks prevent cleanup from deleting foreign or shared resources.
- Rebuild, snapshot/restore, clone, destroy, and uninstall have behavior tests and live evidence for every supported runtime.
- The existing generic compatible-endpoint flow and current Ollama, vLLM, and NVIDIA NIM paths do not regress.

## Constraints and non-goals

The first supported release does not include:

- arbitrary model URLs, unverified GGUF files, or runtime `-hf` model resolution;
- a custom llama.cpp fork;
- CPU-only, Metal, Vulkan, ROCm, Windows-native, or macOS-native execution;
- multi-GPU, distributed, multi-model, router, hot-swap, or multi-tenant serving;
- server-side agents, MCP, filesystem tools, shell tools, Web UI, or public ingress;
- embeddings, reranking, multimodal input, or general llama.cpp flag passthrough;
- OpenAI Responses API compatibility claims;
- automatic conversion from another model format to GGUF;
- a product-support claim based only on generic compatible-endpoint reachability.

These items may use the self-managed compatible-endpoint path or a later accepted issue.

## Decisions required before activation

- Select the first model, exact GGUF file, quantization, chat template, context, and license.
- Select the first `linux/amd64` and `linux/arm64` NVIDIA host profiles.
- Set measurable startup, response, memory, and recovery thresholds for each tuple.
- Confirm the #7744 dependency and protected E2E evidence required for later Podman activation.
- Select the owned image registry path and publication workflow.
- Confirm the exact cooperative fingerprint signals.

## Related work

- [Discussion #7636](https://github.com/NVIDIA/NemoClaw/discussions/7636): managed inference serving specifications.
- [#7407](https://github.com/NVIDIA/NemoClaw/issues/7407): versioned system-readiness report and qualifications.
- [#7744](https://github.com/NVIDIA/NemoClaw/issues/7744): buildless onboarding and runtime-neutral Docker/Podman lifecycle.
- [#6401](https://github.com/NVIDIA/NemoClaw/issues/6401): onboarding architecture.
- [#2412](https://github.com/NVIDIA/NemoClaw/issues/2412) and [PR #5505](https://github.com/NVIDIA/NemoClaw/pull/5505): current self-managed llama.cpp/GGUF path.
- [#7209](https://github.com/NVIDIA/NemoClaw/issues/7209): generic compatible-endpoint validation record.
- [PR #498](https://github.com/NVIDIA/NemoClaw/pull/498): closed incomplete custom-fork provider attempt. This epic uses upstream llama.cpp and is not a continuation of that fork.

- [#8151](https://github.com/NVIDIA/NemoClaw/issues/8151) and [PR #8152](https://github.com/NVIDIA/NemoClaw/pull/8152): merged declarative serving catalog/compiler foundation.
- [#8161](https://github.com/NVIDIA/NemoClaw/issues/8161) and [PR #8167](https://github.com/NVIDIA/NemoClaw/pull/8167): merged cooperative existing-server attachment.
- [#8181](https://github.com/NVIDIA/NemoClaw/issues/8181) and [PR #8183](https://github.com/NVIDIA/NemoClaw/pull/8183): merged typed declarative llama.cpp catalog contract.
- [#8173](https://github.com/NVIDIA/NemoClaw/issues/8173) and [PR #8224](https://github.com/NVIDIA/NemoClaw/pull/8224): merged production repository-owned llama.cpp YAML recipe and preset.
- [#8231](https://github.com/NVIDIA/NemoClaw/issues/8231): owned multiarchitecture llama.cpp server image publication and GPU evidence.
- [PR #8060](https://github.com/NVIDIA/NemoClaw/pull/8060): merged provider-neutral host-local inference receipts.
- [PR #8058](https://github.com/NVIDIA/NemoClaw/pull/8058): separate broader provider lifecycle work; not a dependency of llama.cpp slice 6.

## Definition of done

Close this epic only when all accepted child issues are complete and the exact shipped artifacts pass the full acceptance criteria above. Generic compatible-endpoint connectivity, unit tests, or one successful prompt do not complete the epic.