NVIDIA / NVIDIA/nvbench

Associate parameters with stopping criterion to allow stateful CLI parsing and improved diagnostics

Open
#153 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Cuda
Stars
927
Forks
123
Avg merge
2d 10h
Merged PRs (30d)
2

Description

The custom stopping criterion introduced in #151 stores all parameters for all criterion implementations in a single map. This requires that all parameters must be uniquely named across implementations along with degraded error handling capabilities while parsing CLI arguments.

Each instance of stopping_criterion should hold its own params map, and the CLI parser should check that any arguments relating to these criterion are valid for the selected implementation.

The existing stopping criterion parameters are all documented as follows: Applies to the most recent --benchmark, or all benchmarks if specified before any --benchmark arguments. The new stopping criterion framework should support this pattern as well to allow individual benchmarks to be run with different tolerances.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the stopping_criterion implementation and the CLI parser entry point. Trace how parameters are currently stored and applied to benchmarks, then define the validation path for the selected criterion. Done means each criterion owns its parameters, invalid arguments are diagnosed clearly, and per-benchmark or global tolerance scoping works as described.

Written by the indexing model from the issue text.

Assessment

Tech stack
cli
Domain
cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.