hashgraph / hashgraph/solo-weaver

feat(cli): gate preview commands and flags behind a single manifest so unreleased surface stays out of --help and user docs

Open
#1,014 0 comments 0 reactions 0 assignees View on GitHub
New Feature Pending Triage
Dominant language
Go
Stars
3
Forks
0
Avg merge
3d 5h
Merged PRs (30d)
47

Description

## Problem

Features land on `main` and ship in a release before we have finished testing them. Today there is no way to say "this exists, but it is not ready for end users" — anything merged is immediately discoverable in `solo-provisioner --help` and documented in `docs/quickstart.md` alongside stable surface. Operators find a command, run it, and hit behavior we have not signed off on.

We already hide individual flags, but only by hardcoding it at the registration site:

```go
// cmd/cli/commands/kube/cluster/install.go
common.FlagNodeType().SetVarPHidden(installCmd, &flagNodeType, false)

// cmd/cli/commands/root.go
_ = rootCmd.PersistentFlags().MarkHidden(common.FlagSkipHardwareChecks().Name)
```

That has three problems for this use case:

- The decision is scattered across the command tree, so there is no single place to answer "what is currently hidden?" or to flip something visible when it graduates.
- Graduating a feature means editing Go code in N files instead of one line of data.
- It says nothing about the docs. `docs/quickstart.md` documents hidden surface exactly the same as stable surface.

## Proposed fix

### 1. A single manifest as the source of truth

One embedded data file — e.g. `cmd/cli/commands/common/preview_features.yaml`, loaded with `//go:embed` (same pattern as `pkg/software/infrastructure-catalog.yaml`) — listing command paths and flags that are not yet released to end users:

```yaml
# Surface that exists on main but is not ready for end users.
# Removing an entry here is what "ships" the feature — see docs/dev/preview-features.md.
commands:
- path: "network shape"
reason: "tc HTB classes not validated on multi-NIC hosts"
issue: 784
- path: "consensus migration soak"
reason: "soak watcher pending UAT"
issue: 0
flags:
- command: "block node install"
name: "firewall-config-file"
reason: "declarative firewall input still in review"
issue: 1007
```

At startup, after `rootCmd` is fully assembled, walk the tree once and set `cmd.Hidden = true` / `flag.Hidden = true` for every match. Fail loudly (test, not runtime) on an entry that matches nothing, so the manifest cannot rot into a list of stale paths.

### 2. Hidden, not disabled — with an explicit escape hatch

Cobra's `Hidden` only removes an entry from help output; the command still parses and runs. That is exactly what we want: QA and operators-in-the-loop can still exercise the feature by typing the full path, while end users never discover it.

Add `SOLO_PROVISIONER_PREVIEW=1` (env var, not a flag — a flag would itself show up in `--help`) that un-hides the whole manifest for that invocation, so testers get a complete `--help` without a special build.

Worth deciding during triage whether we also want a second, stricter tier — `blocked`, where invoking the command errors with "preview feature; set `SOLO_PROVISIONER_PREVIEW=1` to enable" — for anything genuinely unsafe to run. Hidden-but-runnable is the right default; blocked should be rare.

### 3. Hiding it from the docs

This is the part that has no built-in mechanism, so recommending the approach explicitly. Our docs are hand-written Markdown read directly on GitHub (no generator, no docs site), so there is nothing that automatically follows `Hidden`. Recommended combination:

**a. A separate, unlinked `docs/preview/` tree.** Preview commands get real documentation — testers need it — but it lives in `docs/preview/.md` and is *not* linked from `README.md`, `docs/quickstart.md`, or the command index. Discoverable to anyone who needs it, invisible on the normal reading path. This composes directly with #1013: graduating a feature is "move `docs/preview/network-shape.md` → `docs/commands/network-shape.md`, add it to the index, delete the manifest entry" — one PR, three mechanical edits.

**b. A CI guard that ties the manifest to the docs.** A unit test (runs in `flow-pull-request-checks`) that reads the manifest and greps the *user-facing* docs tree — everything under `docs/` except `docs/preview/` and `docs/dev/` — for each hidden command path and flag name, failing if one appears. This is what actually prevents drift: it makes it impossible to document a preview feature in the quickstart by accident, without needing a docs generator.

**c. A golden `--help` snapshot test.** Pin the visible command/flag surface of `--help` (top level and one level down) in a testdata file. Any PR that exposes new surface has to update the golden file, which makes accidental exposure a visible, reviewable diff rather than something noticed after release.

Alternatives considered, and why not:

- *Markdown region markers* (`` … ``) stripped at publish time — needs a publish step we do not have; docs are read straight from the repo.
- *Generating command docs from Cobra* (`GenMarkdownTree`) so hiding is automatic — genuinely the best long-term answer, since it would make `Hidden` authoritative for docs too and would retire the `CLAUDE.md` rule-6 drift problem entirely. But it is a much bigger change than this issue, and it fights the hand-written, example-heavy style of the current docs. Worth a separate issue; not a prerequisite.

### 4. Graduation workflow

Document the lifecycle in `docs/dev/preview-features.md`: add to the manifest when merging an untested feature (with reason + tracking issue), remove the entry + move the doc + update the golden help file when it passes UAT. The manifest doubles as a standing list of "merged but not released" surface, which is useful on its own for release notes.

## Acceptance

- [ ] A single embedded manifest file lists preview commands and flags, with a reason and tracking issue per entry.
- [ ] Commands and flags named in the manifest are absent from `solo-provisioner --help` and from every subcommand's help, but still parse and run when invoked explicitly.
- [ ] `SOLO_PROVISIONER_PREVIEW=1` restores the full help output for that invocation.
- [ ] A test fails when a manifest entry matches no command or flag in the tree (no stale entries).
- [ ] A test fails when a manifest-listed command path or flag name appears in the user-facing docs tree.
- [ ] A golden `--help` snapshot test pins the visible surface, so newly exposed commands/flags show up as a reviewable diff.
- [ ] Preview features are documented under `docs/preview/`, not linked from `README.md` or the quickstart/command index.
- [ ] `docs/dev/preview-features.md` documents how to add an entry and how to graduate one.
- [ ] Shell completion does not offer hidden commands (verify — Cobra's completion respects `Hidden`, but confirm for flags too).

### Related Issues

* Depends on #1013 for the `docs/commands/` + `docs/preview/` layout (this issue can land first with a flat `docs/preview/`, but the graduation workflow reads better once the split exists).

Contributor guide

Open the contributing guide

Research direction

Start by reading cmd/cli/commands/root.go and the existing flag registration examples in cmd/cli/commands/kube/cluster/install.go, then compare the embedded-data pattern in pkg/software/infrastructure-catalog.yaml. Review the docs/quickstart.md layout and the flow-pull-request-checks context before choosing the manifest and test locations. Done means the manifest, preview escape hatch, documentation guard, help snapshot, stale-entry test, and graduation documentation satisfy the listed acceptance checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, documentation, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.