NVIDIA-NeMo / NVIDIA-NeMo/DataDesigner

Epic: Async Scheduling Resource Metadata, Admission, and Observability

Open
#645 0 comments 0 reactions 1 assignee View on GitHub

@eric-tramel is already working on this.

Since May 13, 2026.

plan task
Dominant language
Python
Stars
2.2k
Forks
211
Avg merge
2d 6h
Merged PRs (30d)
40

Description

Priority Level

High

Epic Scope

Track implementation and quality gates for the async scheduling architecture. This issue is no longer the architecture specification; the plan files are the source of truth.

Source of Truth

Architecture, UML alignment, contracts, terminology, and cross-cutting decisions live in PR #658 under plans/645/, especially:

  • plans/645/README.md
  • plans/645/architecture.md
  • plans/645/contracts.md
  • plans/645/task-admission.md
  • plans/645/request-admission.md
  • plans/645/capacity-model.md
  • plans/645/observability.md
  • plans/645/benchmark-plan.md
  • plans/645/migration-and-cleanup.md
  • plans/645/issue-map.md

If this issue and plans/645 disagree, update the plan first and keep this issue as implementation tracking only.

Implementation Lanes

  • Metadata lane: #641 -> #646 -> #653 -> #652.
  • Scheduler admission lane: #646 -> #644 -> #647.
  • Request admission lane: #644 -> #654 -> #657 -> #635.
  • Correlation lane: #657 + #635 + #647 -> #648.
  • Benchmark harness lane: #644 + #654 -> #649.
  • Documentation lane: #652 + #654 + #657 + #648 + #649 -> #660.
  • Policy lane: #649 -> #650 -> #651.

Epic PRs should target epic/645-async-scheduling while the epic branch is active unless the PR explicitly documents a different target.

Epic Quality Gates

  • Generator/plugin metadata lands behind the SchedulingMetadata contract and does not expose scheduler internals.
  • Scheduler grouping consumes resolved metadata through the plan-defined bridge, and the legacy hint resolver path is removed or collapsed with no independent introspection fallback.
  • Task admission uses an explicit lease boundary before worker spawn, with root/from-scratch and downstream work visible to the same admission path.
  • Request admission is a distinct model-call-time layer with request leases, exact-once release, adaptive provider/model/domain state, and no durable Throttle* production names.
  • Capacity vocabulary and snapshots distinguish engine, row-group, task-admission, request-admission, static provider caps, adaptive request caps, and transport/session pools.
  • Scheduler/request telemetry remains generic and independently useful, with correlation added through the plan-defined read-only bridge.
  • #649 produces reusable benchmark artifacts before follow-on policy work relies on measured fairness/throughput claims.
  • Final docs explain the two-stage model and capacity/observability surfaces without reintroducing removed names as current APIs.
  • Repo search gates pass for stale production/current-doc terminology called out in plans/645/migration-and-cleanup.md.
  • Existing async scheduler, fair queue, request-admission, end-to-end async engine, docs, and recipe validations continue to pass as applicable to each PR.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.