[Linux][Inference] Add generic managed-vLLM profiles for Muse Glimmer and Nemotron 3.5 Lightning
- Dominant language
- TypeScript
- Stars
- 22.5k
- Forks
- 3.1k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 715
Description
## Summary
Decide whether NemoClaw should add explicit, experimental managed-vLLM profiles for Muse Glimmer 30B and NVIDIA Nemotron 3.5 Lightning on generic Linux hosts with NVIDIA GPUs.
PR #9537 exposed the current behavior: an explicit `NEMOCLAW_VLLM_MODEL` selection stops when the catalog has no compatible Linux preset, even when the model can have a compatible runtime recipe.
## Current behavior
- DGX Spark has explicit Muse Glimmer and Lightning profiles.
- Generic Linux has no matching presets for these models.
- The resolver stops before download or runtime mutation.
- `nemoclaw profiles list` therefore reports no compatible Linux profile for these selections.
## Proposed product decision
Authorize implementation of two explicit-only, Experimental Linux profiles. Do not change any automatic platform default.
Each profile must use the accepted managed-inference catalog and resolver. A new model or host profile must be a catalog change, not a model-specific CLI branch.
## Compatibility boundary
- Linux amd64 and Linux arm64 require separate runtime recipes and immutable image digests.
- Muse Glimmer must not reuse an ARM64-only candidate image on amd64.
- Lightning on Ampere must use its qualified W4A16 execution contract, including the required MoE backend.
- Host admission must evaluate architecture, compute capability, GPU memory, runtime availability, and recipe readiness before download.
- A profile without an accepted runtime and hardware evidence must remain unavailable.
## Lifecycle and security
- Reuse the existing managed vLLM ownership receipt, bearer authentication, loopback listener, route, recovery, and cleanup contracts.
- Keep Hugging Face credentials out of command output, logs, receipts, and sandbox state.
- An explicit incompatible selection must stop before acquisition, container creation, route mutation, or sandbox mutation.
- Existing DGX Spark and DGX Station defaults must not change.
## Validation boundary
- Catalog schema and reference validation.
- Resolver tests for explicit match, incompatibility, ambiguity, and no-match behavior.
- `nemoclaw profiles list` output and incompatibility reason tests.
- Managed vLLM lifecycle and recovery tests.
- Physical evidence for every architecture, GPU family, model, runtime image, and agent combination claimed as tested.
- Brev evidence may establish one tested Linux environment. It does not establish general Linux support.
## Acceptance criteria
- [ ] Maintainers record ownership, lifecycle, compatibility, security, and validation expectations.
- [ ] Maintainers approve the initial Linux architecture and GPU matrix.
- [ ] Every enabled profile pins the model revision and runtime image digest.
- [ ] Explicit selection resolves through catalog data without a model-specific code branch.
- [ ] Unsupported or unqualified combinations stop before effects.
- [ ] Automatic defaults remain unchanged.
- [ ] Documentation distinguishes Experimental, tested, compatible, and supported states.
Related: #8379, #8384, #8385, and PR #9537.
Signed-off-by: Prekshi Vyas
Contributor guide
Research direction
Start by reading PR #9537 and the existing managed-inference catalog, catalog schema validation, and resolver tests. Trace how explicit selections reach `nemoclaw profiles list`, then review the managed vLLM lifecycle and recovery tests. Done means approved architecture and GPU coverage, pinned model and image data, pre-effect rejection of unsupported combinations, unchanged defaults, and updated state documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux, typescript
- Domain
- ai, infrastructure, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100