NVIDIA / NVIDIA/TensorRT

H100 receipts pack: M≪N work-shrink exact retrieval shows 308× lower J/query at N=20M (public-safe)

Open
#4,672 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Module:Performance
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Hi TensorRT team — I’m posting this as a routing request to the right CUDA/performance or inference-performance integration owner.

I have a public-safe H100 results pack (I will attach a zip to this issue). It contains:

  • raw benchmark JSON outputs + measured energy receipts (NVML / nvidia-smi sampling)
  • explicit PASS/FAIL gates
  • a compact one-pager summary (summary_public.json) + schema + a tiny validator script
  • the Python harness used to produce the key measurement (measurement side is not a black box)

Headline result (from a single H100, short steady window; exact top‑1 check):
N=20,000,000 candidates, query_len=256

  • full_scan_top1: p95 ≈ 37.523 ms, energy/query ≈ 4.46297 J
  • range_scan_top1 (M≪N work‑shrink/routing): p95 ≈ 0.11414 ms, energy/query ≈ 0.0144809 J
    => ~308× lower J/query and ~300× lower p95 latency, with top‑1 exactness still matching.

Why I think this matters:
This is a scaling regime shift. The baseline cost scales with N, while the routed path scales with M≪N. At some N the baseline becomes infeasible (OOM wall for explicit NxN fp16 materialization), while the routed path still runs.

How to verify quickly after attaching the zip:

  1. Open README.txt in the zip (it points to the exact JSON field paths).
  2. Check the main record:
    prototypes/prototype_ctdr_landauer_lab/benchmarks/results/gpu_2025-12-19/joules_query_prefix_range.json
    Look for:
    • delta.energy_per_query_ratio_full_over_range
    • passfail.range_scan.pass == true
    • correctness.range.ok == true
  3. Optional: validate the pack summary (no external deps):
    python partner_packet_nvidia/public_teaser/pack_tools/validate_summary_public.py partner_packet_nvidia/public_teaser/assets/summary_public.json

What I’m asking:
Who is the right person/team to evaluate this (CUDA perf / inference perf integration)? If you can route me, I can share a short runbook; I can provide full reproducible harness + implementation details.

(For clarity: the pack contains no kernel source/PTX/SASS.)

CTDR_public_pack_20251219.zip

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Open README.txt in the attached pack, then inspect prototypes/prototype_ctdr_landauer_lab/benchmarks/results/gpu_2025-12-19/joules_query_prefix_range.json and run the provided summary validator. Done means the results and PASS/FAIL fields are verified and the request is routed to the appropriate CUDA or inference-performance owner.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.