[cli-consistency] CLI Consistency Issues - 2026-09-16
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 541
- Avg merge
- 5h 48m
- Merged PRs (30d)
- 773
Description
### Summary
Inspection of the `gh aw` CLI help output (`/tmp/gh-aw/agent/all-help.txt`, run 35101351746) and comparison against `docs/src/content/docs/setup/cli.md` surfaced **3 issues**: 1 high, 2 medium.
**Severity breakdown**
- 🔴 High: 1
- 🟡 Medium: 2
- ⚪ Low: 0
### Findings
#### 1. `gh aw edit --help` documents a `--no-merge` flag that doesn't exist (High)
**Affected command:** `gh aw edit`
**Exact quoted CLI output** (`/tmp/gh-aw/agent/help-output/edit.txt`):
```
Experimental: edit schema-validated workflow frontmatter and recompile its generated file.
The workflow-id may be a workflow name, a Markdown filename, or a path. Changes are
validated before writing. Workflows managed by a source: declaration can be edited
locally; by default, future updates will merge in those local changes (use --no-merge to override).
```
But the `Flags:` section of the same help output has no `--no-merge` entry:
```
Flags:
--add stringArray Append a value to a list (path=value)
--add-import stringArray Append a workflow import path
--add-skill stringArray Append a workflow skill
-d, --dir string Workflow directory (default: $GH_AW_WORKFLOWS_DIR or .github/workflows)
--dry-run Validate changes without writing or compiling
-h, --help Show help for gh aw edit
--remove stringArray Remove a value from a list (path=value)
--remove-import stringArray Remove a workflow import path
--remove-skill stringArray Remove a workflow skill
--schedule string Set a schedule using a fuzzy schedule or cron expression; use off to remove it
--set stringArray Set a frontmatter path (path=value)
--unset stringArray Remove a frontmatter path
```
Confirmed in source: `pkg/cli/edit_command.go` registers `set`, `unset`, `add`, `remove`, `add-import`, `remove-import`, `add-skill`, `remove-skill`, `schedule`, `dry-run`, `dir` — there is no `cmd.Flags().Bool("no-merge", ...)` call anywhere in that file. A `--no-merge` flag does exist on `gh aw update` (`pkg/cli/update_command.go:173`), so this looks like help text copied/adapted from the `update` command's documentation without adding the matching flag (or the flag was never implemented) on `edit`.
**Expected:** Either implement a `--no-merge` flag on `gh aw edit` matching the described behavior, or remove/correct the sentence in the `Long` description in `pkg/cli/edit_command.go` (line 29) so it doesn't reference a non-existent flag.
**Actual:** Help text references a flag that produces "unknown flag" if a user tries `gh aw edit --no-merge`.
**Suggested fix:** In `pkg/cli/edit_command.go`, update the `Long` string to remove the `--no-merge` reference (e.g., drop the parenthetical), or add the flag and wire it into the source-managed-workflow merge logic referenced by the surrounding tests (`edit_command_test.go`, which currently only test `--set`/`--unset`/`--dry-run` against source-managed workflows and expect an error, not a `--no-merge` override).
**Priority:** High — actively misleading users of an experimental command about available flags.
---
#### 2. `edit` and `format` commands are undocumented in `docs/src/content/docs/setup/cli.md` (Medium)
**Affected commands:** `gh aw edit`, `gh aw format`
Both commands are registered in the CLI (`pkg/cli/edit_command.go: NewEditCommand`, `pkg/cli/format_command.go: NewFormatCommand`) and appear in `/tmp/gh-aw/agent/help-output/edit.txt` and `/tmp/gh-aw/agent/help-output/format.txt`, but neither has a corresponding `#### `edit`` or `#### `format`` section in `docs/src/content/docs/setup/cli.md`. A full heading scan of the docs file (`grep -n "^#### `" docs/src/content/docs/setup/cli.md`) lists 37 documented commands, none of which is `edit` or `format`.
**Expected:** Every top-level CLI command should have a matching `####` section in `cli.md`, consistent with the pattern used for `fix`, `compile`, `validate`, `lint`, etc. in the "### Building" section.
**Actual:** `edit` and `format` are missing entirely from the docs, so users reading the CLI reference have no way to discover them.
**Suggested fix:** Add `#### `format`` (documenting that it applies all codemods and normalizes YAML frontmatter, similar to `fix --write` + reformatting) and `#### `edit`` (documenting the `--set/--unset/--add/--remove/--add-import/--remove-import/--add-skill/--remove-skill/--schedule/--dry-run/--dir` flags and the experimental/source-managed-workflow caveats) sections under "### Building", matching the style of neighboring entries.
**Priority:** Medium — documentation completeness gap, not a functional bug.
---
#### 3. `format` command lacks a `-v`/`--verbose`-scoped documented example set matching sibling commands (Medium, folded into #2 scope note)
This is a sub-point of Finding #2: because `format` has no docs entry at all, none of its flags (`--dir/-d`) or its relationship to `fix` (it "applies all available codemods" per the help text, which is stronger than `fix`'s "applies a registry of codemods") is explained anywhere in the public docs. Recommend covering this nuance explicitly when adding the `format` section so users understand `format` vs `fix` vs `compile --fix` overlap.
**Priority:** Medium.
### Inspection metadata
- **Commands inspected:** All 37 top-level commands plus subcommands (`add`, `add-wizard`, `audit`, `checks`, `compile`, `completion` [+ `install`/`uninstall`], `deploy`, `disable`, `doctor`, `domains`, `edit`, `enable`, `env` [+ `get`/`update`], `experiments` [+ `list`/`analyze`], `fix`, `forecast`, `format`, `gh`, `graders` [+ `run`], `health`, `help`, `init`, `json-schema`, `lint`, `list`, `logs`, `mcp` [+ `list`/`list-tools`/`inspect`/`add`], `mcp-server`, `models`, `new`, `outcomes` [+ `history`], `pr` [+ `transfer`], `project` [+ `new`], `remove`, `run`, `secrets` [+ `set`/`bootstrap`], `status`, `trial`, `update`, `upgrade`, `validate`, `version`), totalling ~480 pre-collected help files.
- **Date:** 2026-09-16
- **Method:** Read `/tmp/gh-aw/agent/all-help.txt` and per-command files in `/tmp/gh-aw/agent/help-output/`; cross-referenced flag lists, examples, and descriptions against `docs/src/content/docs/setup/cli.md` and the relevant Go source in `pkg/cli/*.go` to confirm actual flag registration for suspected discrepancies.
- **Repository/Run:** github/gh-aw, run 35101351746
> [!WARNING]
>
> Firewall blocked 1 domain
>
> The following domain was blocked by the firewall during workflow execution:
>
> - `proxy.golang.org`
>
> To allow these domains, add them to the `network.allowed` list in your workflow frontmatter:
>
> ```yaml
> network:
> allowed:
> - defaults
> - "proxy.golang.org"
> ```
>
> See [Network Configuration](https://github.github.com/gh-aw/reference/network/) for more information.
>
>
> Generated by [✅ CLI Consistency Checker](https://github.com/github/gh-aw/actions/runs/35101351746) · copilot · auto · 125.6 AIC · ⌖ 6.05 AIC · ⊞ 7.4K · [◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw+is%3Aissue+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Fcli-consistency-checker%22&type=issues)
> - [x] expires on Sep 18, 2026, 5:34 AM UTC-08:00
Contributor guide
Research direction
Start with pkg/cli/edit_command.go and pkg/cli/format_command.go, then compare their help output with docs/src/content/docs/setup/cli.md and neighboring command sections. Run the relevant edit tests in edit_command_test.go and regenerate or inspect CLI help as needed. Done means the documented commands, flags, caveats, and help text agree with the registered behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100