NVIDIA-NeMo / NVIDIA-NeMo/DataDesigner
Resolve deployment-owned vLLM serving plans for Slurm
@nabinchha is already working on this.
Since Aug 15, 2026.
- Dominant language
- Python
- Stars
- 2.2k
- Forks
- 211
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 40
Description
Priority Level
High
Task Summary
Implement the package-owned vLLM resolver that converts one typed deployment declaration plus planner-supplied placement into process, readiness, backend-endpoint, and logical-endpoint records.
Technical Details & Implementation Plan
- Dispatch on the deployment server discriminator with vLLM as the only implemented backend.
- Keep model alias, model source, and optional served model name distinct.
- Resolve tensor parallelism within a node and pipeline parallelism across nodes in a replica group.
- Derive replica groups, ranks, lanes, rendezvous inputs, and lane-head backend endpoints from planner-owned placement.
- Carry per-deployment startup and distributed-initialization deadlines, launch standoff/stagger, queue backpressure, readiness configuration, tokenized extra arguments, and typed environment bindings.
- Reject extra arguments that override compiler/runtime-owned model identity, ports, topology, rendezvous, launch timing, or middleware behavior.
- Emit typed logical-endpoint inputs so the runtime can aggregate readiness and load-balance healthy backends while preserving overload responses.
- Preserve the inspected vLLM runtime version as digest-bound provenance without maintaining a package-version compatibility matrix.
Acceptance criteria
- A single-node deployment resolves deterministically without importing scheduler or shell modules.
- A multi-node deployment derives the expected replica groups, ranks, pipeline parallelism, and lane-head endpoints.
- Two deployments may select different serving images without endpoint or resource identity collisions.
- Invalid node, GPU, tensor-parallel, replica-group, expert-parallel, image-kind, and resolved-behavior combinations fail before submission.
- Default and overridden launch-timing and queue-backpressure values serialize in golden records.
- A rank failure is represented as failure of the coordinated deployment; no per-replica restart contract is introduced.
Out of scope
- Slurm submission or node allocation.
- Shell rendering, process supervision, and cleanup.
- Image building or registry mutation.
- Dynamo, SGLang, serving plugins, and per-replica recovery.
Investigation / Context
This is a feature lane under #850. Portable deployment intent remains separate from backend-specific process resolution so configuration stays declarative.
Agent Plan / Findings
Build against the reviewed shared configuration, image-inspection, placement, and runtime record contracts. Keep the resolver pure with focused single-node and multi-node golden tests.
Dependencies
Blocked by the shared-contract and fake-infrastructure foundation work tracked by #850.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.