NVIDIA-NeMo / NVIDIA-NeMo/DataDesigner

Compile Slurm configuration into deterministic execution plans

Open
#875 1 comment 0 reactions 1 assignee View on GitHub

@andreatnvidia is already working on this.

Since Aug 24, 2026.

task
Dominant language
Python
Stars
2.2k
Forks
211
Avg merge
2d 6h
Merged PRs (30d)
40

Description

Priority Level

High

Task Summary

Resolve strict authored Slurm input and cluster profiles into an immutable, deterministic execution plan before any scheduler submission occurs.

Technical Details & Implementation Plan
  • Load and validate authored run configuration and the selected cluster profile.
  • Resolve Data Designer invocation inputs, deployments, images, resources, mounts, shard ranges, client placement, and output intent into package-owned plan records.
  • Keep authored semantics separate from resolved scheduler/runtime facts.
  • Produce stable plan serialization and digests suitable for pinning in run, shard, and attempt records.
  • Validate cross-field resource, placement, path, and deployment constraints before submission.
  • Provide pure planning entry points that require no Slurm commands or runtime services.
Acceptance criteria
  • The same authored input, profile, and referenced artifacts produce identical plan bytes and digests.
  • Invalid deployment, placement, resource, mount, shard, and destination combinations fail before submission.
  • Plans contain all inputs required by serving, image, launcher/runtime, client-worker, state, and benchmark lanes.
  • Plans contain no credentials, mutable source references, or unresolved environment-dependent defaults.
  • Unit and golden tests cover single-node, multi-node, multi-deployment, sharded, and invalid configurations.
Out of scope
  • Slurm submission or command parsing.
  • Starting model services or the Data Designer client.
  • State persistence, retry, collection, or benchmark execution.
Investigation / Context

This is the configuration-resolution and deterministic-plan lane in #850. #866 and #868 consume planner-supplied placement and resources but do not own authored configuration or plan compilation.

Agent Plan / Findings

Keep planning pure and deterministic so every side effecting lane consumes a reviewed immutable plan rather than re-resolving authored input.

Dependencies

Depends on the public contract in #851, the optional package foundations in #852 and #853, and the shared records in #873.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.