NVIDIA / NVIDIA/TensorRT-LLM

Persist structured checkpoint-I/O policy and runtime-image identity in startup telemetry

Open
#19,173 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Infra
Dominant language
Python
Stars
14.7k
Forks
2.8k
Avg merge
2d 23h
Merged PRs (30d)
489

Description

Motivation

The startup metrics added by #18608 provide model-loading timings, but effective checkpoint-I/O policy currently depends on parsing server logs. Missing logs or multiple primary/draft loading events can leave attribution incomplete. Metric rows also lack the launch-time runtime-image reference, making matched-image comparisons depend on external joins.

Proposed scope

  • Forward the canonical finalized primary checkpoint-I/O status through existing startup metadata and /server_info, including requested/selected/activated/effective and fallback reason. Keep primary and draft attribution separate; unsupported/unobserved paths remain unknown.
  • Prefer structured status in the CI collector, retaining explicitly labeled legacy-log fallback for older runtimes. Add separate timing and policy completeness indicators.
  • Stamp the actual test-container launch reference at Kubernetes/Slurm launch sites, including architecture-correct runtime images, and persist it with provenance. Record an OCI digest only when actually supplied by a digest-pinned launch reference; tagged references do not prove a resolved digest.
  • Add focused tests and document comparison semantics. No new CI stages or model launches.

Non-goals

No policy-allocation or loader behavior changes, native counterfactual eligibility probes, checkpoint fingerprints, slowest-rank timing/collectives, failed-start persistence, or launch-to-first-token instrumentation. Resolving the running digest for mutable image tags is separate infrastructure work.

Related: #18608, #17393. Independent of the 50/50 split in #19170.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Trace the existing startup metadata and /server_info paths, then inspect the CI collector and Kubernetes/Slurm launch sites. Confirm how primary and draft attribution, legacy-log fallback, completeness indicators, and launch-reference provenance are represented; add the focused tests and documentation needed to verify comparison semantics without adding CI stages or model launches.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, python
Domain
infrastructure, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.