[cli-consistency] CLI Consistency Issues - 2026-09-15

### Summary

Automated CLI consistency inspection of `gh-aw` compared the pre-collected help output (`/tmp/gh-aw/agent/all-help.txt`, generated from the built binary) against `docs/src/content/docs/setup/cli.md`. Two documentation gaps were found; no typos, flag-naming inconsistencies, or broken examples were identified in the help text itself.

**Severity breakdown:** 🟠 High: 1 · 🟡 Medium: 1 · ⚪ Low: 0

### Inspection metadata

- **Commands inspected:** all top-level and nested subcommands listed in `gh aw --help` (43 top-level commands, ~480 help-output files captured in `/tmp/gh-aw/agent/help-output/`)
- **Method:** diffed flag tables, usage strings, and examples in `/tmp/gh-aw/agent/all-help.txt` against `docs/src/content/docs/setup/cli.md`; cross-checked MCP server tool registration against `pkg/cli/mcp_server.go`
- **Date:** 2026-09-15
- **Run:** 34974535215

---

### Finding 1 — `gh aw edit` and `gh aw format` commands are undocumented (🟠 High)

**Affected commands:** `edit`, `format`

Both commands appear in `gh aw --help`'s "Development Commands" section and have full dedicated help text, but neither has a corresponding section in `docs/src/content/docs/setup/cli.md`. A repo-wide search confirms zero mentions of `gh aw edit` or `gh aw format` anywhere under `docs/src/content/docs/`.

**Actual CLI output (`gh aw --help`):**
```
Development Commands:
compile Compile agentic workflow Markdown files into GitHub Actions YAML
domains List network domains configured in agentic workflows
fix Auto-fix deprecated agentic workflow fields using codemods (dry-run by default)
format Apply codemods and normalize agentic workflow frontmatter
lint Lint existing .lock.yml workflow files with actionlint only
mcp Manage MCP (Model Context Protocol) servers
validate Validate agentic workflows without generating lock files
```
(`edit` appears under "Setup Commands" in the same top-level listing.)

**`gh aw edit --help`:**
```
Experimental: edit schema-validated workflow frontmatter and recompile its generated file.

The workflow-id may be a workflow name, a Markdown filename, or a path. Changes are
validated before writing. Workflows managed by a source: declaration can be edited
locally; by default, future updates will merge in those local changes (use --no-merge to override).

Edits that change nothing leave the workflow untouched. When frontmatter does change it is
re-serialized, so YAML comments, key ordering, and quoting styles are not preserved.

Usage:
gh aw edit [path: value] [flags]

Examples:
gh aw edit repo-assist "max-turns: 20"
gh aw edit repo-assist --schedule "every 6h"
gh aw edit repo-assist --set model=small --unset engine.model
gh aw edit repo-assist --add-import shared/common.md
gh aw edit repo-assist --add-skill shared/review

Flags:
--add stringArray Append a value to a list (path=value)
--add-import stringArray Append a workflow import path
--add-skill stringArray Append a workflow skill
-d, --dir string Workflow directory (default: $GH_AW_WORKFLOWS_DIR or .github/workflows)
--dry-run Validate changes without writing or compiling
-h, --help Show help for gh aw edit
--remove stringArray Remove a value from a list (path=value)
--remove-import stringArray Remove a workflow import path
--remove-skill stringArray Remove a workflow skill
--schedule string Set a schedule using a fuzzy schedule or cron expression; use off to remove it
--set stringArray Set a frontmatter path (path=value)
--unset stringArray Remove a frontmatter path
```

**`gh aw format --help`:**
```
Apply all available codemods and normalize YAML frontmatter in agentic workflow files.

Formatting uses two-space indentation and deterministic field ordering while retaining comments
and Markdown content. If no workflows are specified, all Markdown files in .github/workflows are
formatted.

The workflow-id is the basename of the Markdown file without the .md extension.
You can provide either the workflow-id (e.g., 'ci-doctor') or the full filename (e.g., 'ci-doctor.md').

Usage:
gh aw format [workflow]... [flags]

Examples:
gh aw format
gh aw format my-workflow
gh aw format --dir custom/workflows

Flags:
-d, --dir string Workflow directory (default: $GH_AW_WORKFLOWS_DIR or .github/workflows)
-h, --help Show help for gh aw format
```

**Expected vs actual:** Every other command in the "Development Commands" and "Setup Commands" groups (`compile`, `domains`, `fix`, `lint`, `mcp`, `validate`, `add`, `add-wizard`, `deploy`, `doctor`, `env`, `init`, `new`, `remove`, `secrets`, `update`, `upgrade`) has a `####`-level section under `## Commands` in `cli.md`. `edit` and `format` are the only two exceptions — despite `edit` being referenced in an ADR (`docs/adr/55475-schema-validated-workflow-frontmatter-edit-command.md`) and `format` in another (`docs/adr/59971-add-frontmatter-format-command.md`), neither ADR content made it into the user-facing CLI reference page.

**Suggested fix:** Add `#### \`edit\`` (likely under a "Building" or new "Editing" subsection, marked experimental per its own help text) and `#### \`format\`` (fits naturally next to `#### \`fix\`` under "### Building") sections to `docs/src/content/docs/setup/cli.md`, each with a short description, example block, and **Options:** line, matching the style of neighboring commands.

**Priority:** High — these are non-trivial, user-facing commands with unique flag surfaces (`--set`/`--unset`/`--add`/`--remove` path syntax for `edit`) that are currently discoverable only via `--help`, not the reference docs.

---

### Finding 2 — `mcp-server` "Available Tools" list in docs is missing `audit-diff` (and is stale relative to the code) (🟡 Medium)

**Affected command:** `mcp-server`

`docs/src/content/docs/setup/cli.md` states:

```
**Available Tools:** status, compile, logs, audit, checks, mcp-inspect, add, update, fix
```

That's 9 tools. However, `pkg/cli/mcp_server.go` registers 10 tools:

```go
registerStatusTool(server)
registerCompileTool(server, execCmd, manifestCacheFile)
registerLogsTool(server, execCmd, actor, validateActor)
registerAuditTool(server, execCmd, actor, validateActor)
registerAuditDiffTool(server, execCmd, actor, validateActor) // <-- missing from docs
registerChecksTool(server)
registerMCPInspectTool(server, execCmd)
registerAddTool(server, execCmd)
registerUpdateTool(server, execCmd)
registerFixTool(server, execCmd)
```

The `audit-diff` tool (registered via `registerAuditDiffTool` in `pkg/cli/mcp_tools_privileged.go`) exposes the `gh aw audit` multi-run diff/comparison functionality as its own MCP tool, distinct from the single-run `audit` tool, but is not listed anywhere in the "Available Tools" line.

**Expected vs actual:** Docs list 9 tool names; the running server exposes 10 MCP tools, name `audit-diff`.

**Suggested fix:** Update the line in `cli.md` to:
```
**Available Tools:** status, compile, logs, audit, audit-diff, checks, mcp-inspect, add, update, fix
```

**Priority:** Medium — this is a discoverability gap for MCP clients/users relying on the docs rather than introspecting the server directly; the tool works correctly, only the doc enumeration is incomplete.

---

Areas checked and found consistent (no issues)

- Global flags (`--banner`, `-h/--help`, `-v/--verbose`, `--version`) — consistent across all ~480 captured help files.
- Short-flag/long-flag pairing (e.g. `-d/--dir`, `-e/--engine`, `-r/--repo`, `-j/--json`, `-f/--force`, `-y/--yes`) — no command assigns a different short flag to the same long flag name elsewhere.
- `--stop-after` / `--no-stop-after` pairing — every command exposing one exposes the other.
- `compile`, `validate`, `lint` flag sets and descriptions — match `cli.md` `**Options:**` lines exactly.
- `logs`, `audit`, `health`, `forecast`, `checks`, `models`, `experiments`, `outcomes`, `graders` flags — match docs.
- `add`, `add-wizard`, `deploy`, `update`, `upgrade`, `trial`, `init` flags — match docs, including org-mode (`--org`/`--repos`/`--yes`) flag sets.
- `enable`, `disable`, `remove`, `domains`, `mcp`, `mcp inspect`, `mcp add`, `mcp list-tools`, `pr transfer`, `hash-frontmatter`, `json-schema`, `env get`, `env update`, `new`, `secrets`, `secrets bootstrap`, `secrets set`, `doctor`, `project`, `project new` — match docs.
- No typos found in help text (checked common misspelling patterns: "recieve", "seperate", "occured", "existant", "paramter", "wich", "adress", "accross", "compatability").
- Cross-references in "Advanced and enterprise setup", "Debug Logging", "Smart Features", and "Troubleshooting" sections were spot-checked against actual behavior and found accurate.

> Generated by [✅ CLI Consistency Checker](https://github.com/github/gh-aw/actions/runs/34974535215) · copilot · auto · 105.7 AIC · ⌖ 6.12 AIC · ⊞ 7.4K · [◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw+is%3Aissue+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Fcli-consistency-checker%22&type=issues)
> - [x] expires on Sep 17, 2026, 5:34 AM UTC-08:00

Start with docs/src/content/docs/setup/cli.md and compare its neighboring command sections with the supplied gh aw edit and gh aw format help output. Check pkg/cli/mcp_server.go and pkg/cli/mcp_tools_privileged.go to confirm the audit-diff tool name. Done means the two command sections and the complete 10-tool list match the current CLI and MCP registration details.

Written by the indexing model from the issue text.