NVIDIA / NVIDIA/daqiri

[FEA] Generate example and benchmark configs deterministically

Open
#302 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
C++
Stars
36
Forks
11
Avg merge
3d 4h
Merged PRs (30d)
25

Description

Is your feature request related to a problem? Please describe.

examples/ contains many near-duplicate YAML configurations, and the repository has accumulated separate scripts that generate or prune overlapping Spark benchmark variants. Configuration rules have also historically been spread across the C++ decoder, prose documentation, and copied examples. This makes matrix expansion expensive and allows misspelled keys to fall back silently to defaults.

The configuration generator should consolidate those variants without introducing a second definition of what DAQIRI accepts.

Architectural decision

parse_network_config is the sole authority for the daqiri.cfg language.

An earlier implementation added a hand-written JSON Schema alongside the C++ parser. That created two independently maintained validators for the same required fields, types, enums, ranges, and unknown-key policy. Testing a finite generated corpus through both could detect some drift, but could not prevent it. The schema was therefore removed.

Python profile classes may validate generator-specific inputs and relationships so the CLI can produce useful errors. They do not validate arbitrary DAQIRI YAML. A future editor/documentation schema is acceptable only if it is generated from the runtime parser's source metadata or is itself consumed by the runtime; it is outside this issue.

Solution

  1. Add scripts/gen_daqiri_config.py, a deterministic generator for raw Ethernet, UDP, TCP, and RoCE configurations. The same inputs must produce byte-identical YAML.
  2. Expose the existing C++ parser through a parse-only daqiri_config_validate executable. It performs no hardware initialization.
  3. Make the C++ parser reject unknown keys, invalid types, unsupported enum values, out-of-range integers, and malformed nested structures rather than silently applying defaults.
  4. Exercise retained and generated configurations through that executable, including focused rejection cases.
  5. Replace repetitive matrix YAML files and the two bespoke Spark transformation scripts with shared generator invocations, while retaining hand-written teaching and application configurations.
  6. Install the generator package, CLI, and validation executable.

The generator and validator are deliberately separate commands: generation constructs a requested profile; acceptance of emitted or hand-written DAQIRI YAML is decided by the same parser used by applications.

CI coverage

The normal pull-request workflow runs portable Python tests and builds the C++ validator with DAQIRI_ENGINE="ibverbs". It validates all retained and generated configurations except those that explicitly require DPDK. This keeps the PR gate reasonably fast and avoids relying on the hosted runner's packaged DPDK, which is older than the APIs used by DAQIRI.

The existing release-container workflow builds the project's patched DPDK plus ibverbs and runs the same retained/generated checks without the DPDK exclusion. This supplies full-engine release coverage, but it is intentionally not described as a pull-request gate.

If faster full-engine images become available, moving DPDK validation into the PR workflow can be considered separately.

Acceptance criteria

  • Generated output is deterministic across independent Python processes.
  • Retained teaching/application configurations parse with parse_network_config after typed placeholders are materialized.
  • The generated transport, transform, queue-shape, and split-role matrix parses with parse_network_config for the engines available in the build.
  • Focused invalid configurations fail cleanly through the C++ validator.
  • Unknown configuration keys are rejected by the runtime parser.
  • Repetitive generated YAMLs and the bespoke Spark pruning scripts are removed.
  • Documentation distinguishes profile construction from authoritative runtime validation.

Alternatives considered

  • Hand-maintained YAML variants: do not scale and already led to multiple one-off generators.
  • More superset-and-prune scripts: continue duplicating benchmark-specific transformation logic.
  • A hand-written JSON Schema beside the C++ parser: rejected because it creates two validators that can drift.
  • Schema as the runtime source of truth: coherent, but would require runtime schema validation or generated C++ decoding and is substantially larger than this issue.

Hand-written, commented examples remain where they teach concepts; only repetitive matrix variants should be generated.

Related: #279 can build an interactive front-end over this generator and the C++ validation command. PR #298 may consume the shared profiles rather than independently reconstructing DAQIRI configuration mappings.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating parse_network_config, the existing configuration-generation and Spark transformation scripts, and the C++ build and test entry points. Review the retained examples and the proposed scripts/validator boundaries before making changes. Done means deterministic generated YAML, parser-backed acceptance and rejection checks, removed repetitive variants, installed tools, and documentation that distinguishes profile construction from runtime validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python, spark, yaml
Domain
build-system, cli, testing-qa, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.