NVIDIA / NVIDIA/nvcf

fix(stargate): address protocol, lifecycle, recovery, and benchmark audit findings

Open
#1,817 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
218
Forks
72
Avg merge
1d 12h
Merged PRs (30d)
427

Description

Problem

A Stargate maintenance review identified correctness and resource-lifetime defects in protocol handling, proxy retries, registration recovery, and benchmark reporting.

Resolution

Implemented in a four-layer draft PR stack:

Domain Corrections PR
Protocol contracts IPv6 backend parsing, owned and escaped JSON inputs, connection-specific HTTP headers, and direct schema generation with behavior tests #1818
Proxy and relay behavior Prevent duplicate accepted POST retries, reap completed relay tasks, and bound unknown-target metrics #1820
Registration recovery Kubernetes watcher backoff, non-TLS warnings on every failed retry, safe diagnostics, and token-file redaction #1823
Benchmark accuracy Validate completed SSE output, report observed token usage, preserve unknown measurements, and run scenario tests through Bazel #1826

The router deployment uses raw QUIC. Bidirectional-stream priority is intentional, and the proposed stream-fairness correction has been removed from #1820.

The output-timeout defect was already fixed by #1452 on main. Its first-output and subsequent-output deadline regressions were verified as part of the workspace tests.

Verification

  • Bidirectional-priority restoration: all 24 stargate-forwarding tests passed; all-target Clippy passed with warnings denied. Removed the fair-direction helper, enum, and backlog fixture.
  • Direct tracing update: 515 pylon-lib unit tests and 2 public API tests passed; all-target pylon-lib Clippy passed with warnings denied. Retry warnings now use tracing macros at the failure sites; safe formatting remains separate.
  • Prior combined stack validation: 1,611 Cargo workspace tests passed, 3 existing performance tests ignored, workspace Clippy passed, and 6 scoped Bazel targets passed.
  • Changed Rust files passed formatting checks; the stack passed whitespace and fanout validation.

No GPU or live-cluster performance campaign was run. The PRs are draft and have not been merged.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The work is already implemented in draft PRs #1818, #1820, #1823, and #1826; start by reviewing those changes and their scoped tests. Run the named stargate-forwarding, pylon-lib, workspace, and Bazel checks described in the verification section. Done means the listed protocol, retry, recovery, and benchmark findings are covered without introducing the removed fairness change.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, rust
Domain
backend, infrastructure, performance, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.