CLI Architecture Improvement Proposal

CLI Architecture Redesign

Use cases, pain points, and background

• The current CLI has 17+ separate entry points (ng_run, ng_test, ng_collect_rollouts, etc.)
• The commands don't have a consistent implementation: we have a mix of Pydantic config classes (RunConfig, TestConfig), plain functions (dev_test(), dump_config()), some use BaseNeMoGymCLIConfig, some don't ... there is no clear pattern for adding new commands
• CLI documentation page (docs/reference/cli-commands.md) is written manually because there is no consistent way to pull information from code
• Parameters are passed with Hydra syntax: +param=value (not --param value), which is non-standard for CLI
• No tab completion, hard to implement with Hydra config
• Poor discoverability, requires ng_help to see all commands. Once again due to missing tab auto-completion and --help

Description:
Re-implement the CLI with a single entry point and standard CLI conventions.

Design:

• Use Click or similar framework to write the CLI
• All commands must follow the same structure
• Support tab auto-completion
• Have a single entrypoint ng or nemo_gym
• Group the sub-commands into:
  • server: Server management (run, test, test-all, init)
  • data: Data operations (collect, prepare, view)
  • dataset: Dataset registry (upload, download, delete, migrate)
  • config: Configuration utilities (dump, validate)
• Use standard CLI syntax for parameters (not Hydra)
• Auto-generate CLI documentation from docstrings

Example Usage:

# Server commands
ng server run --config-paths config.yaml
ng server test --entrypoint resources_servers/my_server
ng server test-all
ng server init --entrypoint resources_servers/new_server

# Data commands
ng data collect --agent my_agent --input data.jsonl --output rollouts.jsonl
ng data prepare --output-dir data/ --mode example_validation
ng data view --input rollouts.jsonl

# Dataset commands
ng dataset upload --source data.jsonl --registry gitlab
ng dataset download --name my_dataset --registry huggingface
ng dataset delete --name my_dataset --registry gitlab
ng dataset migrate --from gitlab --to huggingface

# Config commands
ng config dump
ng config validate --config config.yaml

# Utility commands
ng version [--json]
ng help [command]

Start by inventorying the existing entry points named in the issue, including ng_run, ng_test, ng_collect_rollouts, and the related configuration classes and functions. Read docs/reference/cli-commands.md and compare its manual command descriptions with the current implementation. Done means a settled plan for one ng or nemo_gym entry point, grouped subcommands, standard options, completion and help, and generated CLI documentation.