awslabs / awslabs/cli-agent-orchestrator
Mid-run human approval gates as a first-class workflow primitive
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 267
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 70
Description
Parent: #583
## Context
The workflow I authored needed **four human decision points inside one run** — approve the extracted claims, approve the adjudication, approve the fix plan, approve the commit — each one a place where a person reads an artifact and the work must not continue without them.
CAO has no primitive for this, so I hand-built it:
- marker files under `state/gates/-.approved`;
- a `finish()` helper that writes a human-readable `NEXT-STEP.md` (what was produced, what to read, the exact command to resume) and then exits;
- `SystemExit(0)` for the exit — because a non-zero exit reports the run as `failed`, which **misrepresents a deliberate, successful stop** as a fault. That distinction cost a round to discover and is invisible in the artifact.
It worked well enough that I would keep the shape. But every author will rebuild it, every author will get the exit-status detail wrong once, and an operator reading `cao workflow status` cannot tell "stopped for a human" from "crashed".
## Gap vs #583
The roadmap's approval is **pre-run**: resolve the plan, present it, user approves, then "run and observe". The target-experience diagram has exactly one approval, before execution.
That is the right model for a short deterministic job, and insufficient for the long-running staged workflows the roadmap explicitly wants to validate against ("a staged product-build workflow", "the existing PR review workflow"). A PR-review workflow that cannot stop for a human between adjudication and fixing is not the workflow anyone wants — the whole point is that a person adjudicates.
## Proposal
Make the mid-run gate a primitive:
```python
gate("plan-approved",
summary="7 fix clusters across 3 waves",
artifacts=["state/artifacts/plan.md"]) # blocks until approved
```
- A run parked at a gate has its own terminal state — `awaiting_approval`, distinct from both `completed` and `failed` — surfaced in `status`, results, and the MCP tools.
- CAO renders the operator view: which gate, what to read, the exact resume command. (This is what I hand-wrote as `NEXT-STEP.md`; the runtime knows all of it.)
- `cao workflow approve ` / `reject [--reason]`, with the decision, actor, and timestamp recorded in the journal so an approval is auditable evidence rather than the presence of a file on someone's laptop.
- Resume from a gate reuses journaled work for everything before it (depends on Milestone 0).
- A rejection is a first-class outcome, not a crash.
## Acceptance criteria
- [ ] A script can declare a mid-run gate with a summary and the artifacts to review.
- [ ] A run stopped at a gate reports `awaiting_approval`, never `failed`.
- [ ] `status` names the pending gate, the artifacts to read, and the resume command.
- [ ] Approval and rejection are explicit commands recorded in the journal with actor and timestamp.
- [ ] Resuming after approval does not re-execute work completed before the gate.
- [ ] A rejected gate terminates the run with a distinct, non-fault outcome.
Contributor guide
Research direction
Start by tracing the workflow run state and resume entry points, then inspect how status, results, MCP tools, and the journal represent terminal outcomes. Review the proposed `cao workflow approve` and `reject` commands and the gate declaration shape. Done means all acceptance criteria are covered, including auditable decisions, distinct outcomes, and reuse of work completed before a gate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devtools
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100