[Epic] Support SGLang as a managed local inference backend

## Outcome

NemoClaw can install and operate SGLang as a first-class managed local inference backend through serving-catalog profiles.

Managed SGLang follows the same product contract as managed vLLM and managed llama.cpp:

- catalog profile discovery and exact resolution;
- installation and immutable artifact acquisition;
- authentication and credential custody;
- private host networking and firewall diagnostics;
- status, logs, start, stop, and restart behavior;
- interruption recovery and onboarding resume;
- ownership-aware rollback and cleanup; and
- evidence-backed activation for each supported profile.

The existing **Other OpenAI-compatible endpoint** path remains available for operator-managed SGLang servers.

A maintainer product decision accepts SGLang as a first-class managed-backend direction for NemoClaw. Opening this epic records that scope. It does not activate a profile or establish a support claim before the applicable gates pass.

## Context

NemoClaw already supports operator-managed SGLang through the custom OpenAI-compatible provider. That path validates the endpoint and routes sandbox traffic through `inference.local`, but the operator owns installation, artifacts, process lifecycle, networking, recovery, and cleanup.

The serving catalog currently has production materialization and lifecycle implementations for managed vLLM and managed llama.cpp. It can describe other backend strings, but Express onboarding rejects a backend that has no registered implementation.

The current owning surfaces include:

- `src/lib/inference/serving/` for catalog resolution, adapter registration, provenance, receipts, and profile listing;
- `src/lib/inference/vllm.ts` and its serving adapters for managed vLLM;
- `src/lib/inference/llama-cpp/` for managed llama.cpp;
- `src/lib/onboard/` for provider selection, preflight, recovery, and sandbox creation;
- `docs/inference/` for provider and local-inference guidance; and
- `test/e2e/` for live product-flow qualification.

This epic is backend-wide. A model-specific SGLang recipe can become an activation profile only after it satisfies this contract.

## Why this matters

SGLang expands NemoClaw's ability to select the inference engine that best matches each model and NVIDIA hardware profile.

Some models and acceleration features become available or perform best in SGLang before equivalent support exists in vLLM or llama.cpp. Examples can include speculative decoding, model-specific parsers, structured tool calling, multimodal inference, and hardware-specific optimizations.

First-class SGLang support would:

- prevent NemoClaw's managed inference roadmap from depending on one serving engine;
- let catalog profiles select a validated backend for each model and host;
- give SGLang users the same authentication, network isolation, recovery, and cleanup guarantees as managed vLLM and llama.cpp;
- convert operator-managed experiments into repeatable product flows; and
- preserve one consistent NemoClaw experience across supported inference engines.

This epic does not make SGLang the default. Each profile must justify its backend through compatibility, capability, performance, security, and lifecycle evidence.

## Problem statement

NemoClaw has no managed SGLang contract for:

- representing an SGLang runtime and model recipe in the serving catalog;
- selecting only profiles compatible with the detected host;
- acquiring exact SGLang images, model snapshots, and optional draft-model snapshots;
- creating and storing a managed bearer credential;
- publishing the server only on host loopback and the validated OpenShell bridge;
- proving API, reasoning, tool-call, streaming, and optional acceleration behavior;
- identifying and reconciling the exact managed container after interruption or host restart;
- preserving foreign or ambiguously owned resources;
- resuming an incomplete installation without changing the selected recipe;
- rolling back a failed installation without deleting reusable or foreign state; or
- uninstalling the backend and credential only when no managed consumer remains.

Without these contracts, a SGLang deployment remains operator-managed. NemoClaw cannot advertise one-command managed SGLang support.

## Desired behavior

For an accepted host and profile, a user can run:

```text
nemoclaw profiles list
nemoclaw onboard --profile
```

NemoClaw must:

1. Resolve one stable preset and recipe with exact provenance.
2. Reject unknown, ambiguous, disabled, or incompatible profiles before runtime mutation.
3. Show the backend, model, immutable runtime image, support state, and estimated downloads before confirmation.
4. Run host, GPU, architecture, storage, port, network, and ownership preflight checks.
5. Acquire exact runtime, target-model, and optional draft-model artifacts.
6. Start one managed SGLang server with the profile's exact argument vector.
7. Keep the raw bearer credential outside the sandbox.
8. Publish the protected endpoint only on loopback and the exact private OpenShell bridge gateway.
9. Register the provider and route sandbox traffic through `https://inference.local`.
10. Persist exact runtime ownership, authentication fingerprint, network, profile, and artifact evidence.
11. Recover or resume from each supported partial state without silently selecting another profile.
12. Remove only resources proven to belong to the managed runtime and current cleanup authority.

## Architecture boundary

Managed SGLang must extend the serving-catalog and adapter contracts. It must not create a parallel onboarding or lifecycle architecture.

- Add a typed SGLang recipe and one registered materializer, preparation implementation, and lifecycle implementation.
- Keep central catalog resolution and onboarding free of model-specific or profile-specific SGLang branches.
- Reuse backend-neutral host-local serving controls only when vLLM, llama.cpp, and SGLang share the same invariant.
- Keep SGLang command construction and SGLang-specific readiness evidence inside the SGLang implementation.
- Do not treat a vLLM container or command as SGLang because both expose an OpenAI-compatible API.
- Use immutable image and model identities. Do not register mutable tags, branches, or unpinned revisions as supported profiles.
- Bind destructive actions to an exact container ID, immutable image, ownership labels, lifecycle receipt, and current transaction authority.
- Keep the serving credential outside catalog YAML, command arguments, logs, receipts, and sandbox state.
- Preserve the custom compatible-endpoint provider for operator-managed servers.

## Product decisions and activation inputs

The backend direction is accepted. Each activation still requires maintainers to define:

- product and engineering ownership;
- the first supported host and architecture matrix;
- the first model and optional draft-model profiles;
- the exact SGLang release and immutable image digests;
- the initial Experimental or supported status;
- the accepted agent matrix;
- Chat Completions and Responses API claims;
- text, vision, reasoning, tool-call, structured-output, and speculative-decoding claims;
- concurrency, context, memory, and storage limits;
- single-host and multi-host scope;
- update, compatibility, vulnerability-response, and deprecation ownership; and
- protected E2E hardware and evidence ownership.

A mutable development image or unmerged upstream branch can support feasibility testing. It cannot satisfy a managed-profile activation gate.

## Capability slices

### Slice 1: Qualify exact SGLang recipes without product activation

Outcome:

- Maintainers can load and inspect an inactive SGLang catalog recipe and run a dedicated qualification harness without making SGLang selectable in production onboarding.

Acceptance evidence:

- The catalog validates stable IDs, exact image digests, exact model revisions, optional draft revisions, argument vectors, download estimates, host selectors, support state, and adapter references.
- Missing adapters, mutable artifacts, ambiguous selectors, incompatible hosts, and unsupported capabilities fail before external mutation.
- The qualification harness records the exact NemoClaw commit, image digest and platform, model revisions, host identity, command, API results, and cleanup result.

Deferred scope:

- Production selection, installation, provider registration, and support documentation.

### Slice 2: Manage one Experimental single-host SGLang profile end to end

Outcome:

- One accepted host can install, operate, recover, and remove one Experimental SGLang profile through normal NemoClaw commands.

Acceptance evidence:

- Installation, bearer authentication, private bridge publication, route registration, status, logs, restart, resume, rollback, and cleanup satisfy this epic.
- Foreign containers, ports, networks, credentials, or receipts remain unchanged.
- A blocked bridge probe fails closed and reports remediation only for a validated narrow topology.
- A physical host completes the accepted direct-API and agent product flow without fallback.

Deferred scope:

- Default selection, additional hardware, additional models, multi-host serving, and capabilities not claimed by the first profile.

### Slice 3: Add profiles only with profile-specific evidence

Outcome:

- Maintainers can add another SGLang model, host, acceleration strategy, or agent without widening existing profile claims.

Acceptance evidence:

- Each profile pins its complete runtime tuple and declares only measured capabilities.
- Existing profile defaults, priority, recovery, and cleanup behavior remain unchanged.
- Each claimed agent and hardware combination has exact live evidence.

Deferred scope:

- Automatic promotion from Experimental and changing the default local backend.

### Slice 4: Activate supported SGLang product status

Outcome:

- NemoClaw documents SGLang as supported only for the exact matrix that passes the production activation gate.

Acceptance evidence:

- Stable upstream artifacts, lifecycle ownership, vulnerability response, compatibility policy, protected E2E, security review, and user documentation pass for the complete advertised matrix.
- Unsupported profiles remain absent or visibly Experimental.

Deferred scope:

- Any model, host, topology, API, or capability outside the accepted matrix.

## Security invariants

- The sandbox must not receive the raw SGLang bearer credential.
- The long-lived server must receive only the credential and host resources required by its profile.
- The managed port must not bind to `0.0.0.0`, `::`, a LAN address, or an internet-facing address.
- The endpoint must bind only to host loopback and the exact validated private OpenShell bridge gateway.
- Missing or ambiguous bridge topology must fail closed without printing a firewall command.
- A validated firewall failure may print only the exact narrow operator command for the proven subnet, gateway, and port.
- Image, target-model, draft-model, command, authentication, network, and container identities must be immutable in the lifecycle receipt.
- Unknown, foreign, stale, or name-reused resources must not be stopped, replaced, or removed.
- Secrets must not appear in command arguments, catalog files, process listings, logs, receipts, diagnostics, or evidence artifacts.
- Failed validation must not register a usable route to an unauthenticated or unverified server.
- Cleanup must remove the bearer key only after the exact managed runtime is removed and no sibling gateway depends on it.

## Observable acceptance examples

### Allowed

- A compatible host selects one accepted profile and confirms installation. NemoClaw acquires the exact artifacts, starts the server, registers the protected route, and completes an agent turn through `inference.local`.
- A completed model download or image pull is reused after an interrupted onboarding session when its exact identity still matches.
- A finalized managed runtime resumes after host restart when its receipt, container, image, command, authentication fingerprint, and network identity still match.

### Denied

- A recipe uses a mutable image tag, branch name, unpinned model revision, or unknown adapter reference. Catalog validation rejects it before profile selection.
- A host does not meet the profile's architecture, GPU, compute-capability, memory, storage, or topology requirements. Preflight stops before the guarded mutation.
- Another process owns the requested port, or a same-name container lacks exact ownership evidence. NemoClaw preserves it and stops.
- A recipe requests an unsupported API, parser, modality, acceleration mode, or privilege. Resolution stops before installation.
- The managed endpoint accepts an unauthenticated request. Installation fails and no usable route remains registered.

### Ambiguous

- Two automatic profiles have the same highest priority. Automatic selection stops; interactive onboarding may list exact choices without marking either recommended.
- Runtime labels match but the receipt, container ID, image, command, authentication fingerprint, or network identity differs. Lifecycle mutation stops and reports the mismatched field.
- Docker or host storage cannot be resolved. The profile follows the accepted fail-closed or explicit-confirmation storage contract; it does not claim sufficient capacity.

### Failure and recovery

- Image or model acquisition fails. NemoClaw keeps verified reusable cache data, records the resumable state, and does not start a partial long-lived server.
- Server startup, readiness, authentication, or route validation fails. NemoClaw removes or stops only the exact transaction-owned runtime and records retained state.
- The process exits during installation or cleanup. The next run classifies the journal and receipt before it mutates runtime state.
- Cleanup cannot prove ownership or remove one resource. NemoClaw preserves the remaining authority and bearer key and reports an incomplete cleanup result.

## Validation and evidence gates

Every activated profile must provide:

- deterministic catalog, resolver, command-construction, receipt, recovery, and cleanup tests;
- negative tests for malformed artifacts, unsafe arguments, credentials, wildcard bindings, foreign ownership, name reuse, topology ambiguity, and partial state;
- exact image digest and platform verification;
- exact target-model and optional draft-model revision verification;
- `/health` and `/v1/models` identity checks;
- authenticated and unauthenticated API boundary checks;
- Chat Completions streaming with system-prompt preservation;
- reasoning and structured tool-call checks when claimed;
- Responses API checks only when claimed;
- speculative-decoding activation and correctness checks when claimed;
- one real agent turn for every claimed agent runtime, with exact provider, model, completion, and no-fallback evidence;
- restart, interruption, resume, rollback, uninstall, and sibling-gateway cleanup evidence;
- firewall-denied and validated-remediation evidence for the managed bridge path;
- existing vLLM, llama.cpp, Ollama, NIM, and custom-endpoint regression coverage; and
- documentation writer and security reviews of the latest PR commit.

Hardware evidence must identify the exact NemoClaw commit, SGLang image digest, model revisions, profile, host class, and result.

## Activation gates

### Experimental profile

An Experimental profile may activate only when:

- maintainers accept its exact reduced scope;
- all artifacts are immutable and distributable;
- complete managed lifecycle and security behavior passes;
- every unsupported capability is named;
- protected live E2E passes on each advertised host and agent; and
- the profile is visibly Experimental and does not replace an existing default.

### Supported profile

A supported profile requires:

- a stable upstream SGLang release and maintained immutable images;
- an accepted compatibility, update, and vulnerability-response policy;
- the complete advertised host, model, agent, API, and capability matrix;
- sustained inference, restart, recovery, cleanup, and upgrade evidence;
- protected release-candidate E2E; and
- evidence-backed setup, operation, security, platform-support, and troubleshooting documentation.

## Constraints and non-goals

- Do not remove or weaken the operator-managed OpenAI-compatible endpoint path.
- Do not make SGLang the default local backend without a separate maintainer decision and comparative evidence.
- Do not make this epic specific to Muse Glimmer or another model.
- Do not register mutable development images or unmerged source branches as supported dependencies.
- Do not copy vLLM or llama.cpp implementation code when a backend-neutral contract already owns the behavior.
- Do not force SGLang-specific arguments into a generic OpenAI-compatible provider.
- Do not enable arbitrary operator arguments in a managed profile without a bounded accepted contract.
- Do not claim vision, speculative decoding, Responses API, multi-host serving, or an agent runtime without exact evidence.
- Do not change `sgl-project/sglang` through a NemoClaw PR.

## Dependencies and related work

- Managed vLLM provides the current host-local catalog, acquisition, bearer, bridge, receipt, recovery, and cleanup reference.
- Managed llama.cpp provides the current second-backend catalog and adapter reference.
- Issue #8712 and its merged implementation define the fail-closed managed private-bridge and narrow firewall-remediation boundary.
- Epic #7744 demonstrates the repository's provider-neutral and activation-gated design principles. Managed SGLang is not a sandbox runtime-provider implementation under that epic.
- Epic #8178 demonstrates the separate product-decision and exact-matrix activation boundary. Native Windows/MXC is not a dependency of managed SGLang.
- Upstream SGLang release, image, and model support can block a profile activation. They do not block inert catalog and qualification work.

## Work tracking

- [ ] Accept the initial host, model, agent, API, capability, and support-status matrix.
- [ ] Add the inactive typed SGLang catalog and adapter contracts.
- [ ] Add an opt-in qualification harness with exact evidence and cleanup.
- [ ] Implement immutable acquisition, preflight, and command construction.
- [ ] Implement bearer authentication and private bridge networking.
- [ ] Implement ownership receipts, status, logs, restart, and reconciliation.
- [ ] Implement interruption recovery, onboarding resume, rollback, and cleanup.
- [ ] Add the first Experimental profile only after its activation gate passes.
- [ ] Add protected physical-hardware E2E for each advertised profile.
- [ ] Publish documentation that matches the activated matrix.
- [ ] Evaluate supported status only after the production activation gate passes.

## Definition of done

This epic closes when NemoClaw has at least one evidence-backed managed SGLang profile; the serving catalog, installation, authentication, networking, lifecycle, recovery, and cleanup contracts pass; operator-managed SGLang remains available; foreign resources and credentials remain protected; protected live E2E passes for the advertised matrix; and documentation matches the shipped support state.

## Category

Inference

## Checklist

- [x] I searched open and closed issues and this is not a duplicate.
- [x] I described the problem and desired behavior.